[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:43:56 dist-test-slave-9gcw krb5kdc[22368](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:43:56 dist-test-slave-9gcw krb5kdc[22368](info): set up 2 sockets
Aug 11 01:43:56 dist-test-slave-9gcw krb5kdc[22368](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 01:43:57.479283 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.012s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:43:57 dist-test-slave-9gcw krb5kdc[22368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:43:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:43:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:43:57.699555 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:36883
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:36883
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:43:58.276765 22381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:43:58.277262 22381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:43:58.277456 22381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:43:58.288619 22381 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:43:58.288991 22381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:43:58.289274 22381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:43:58.289547 22381 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:43:58.289745 22381 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:43:58.309119 22381 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:36883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:36883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:43:58.310956 22381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:43:58.312693 22381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:43:58.335891 22387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:43:58.335852 22390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:43:58.335594 22388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:43:58.337826 22381 server_base.cc:1034] running on GCE node
I20260811 01:43:58.338879 22381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:43:58.340595 22381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:43:58.341845 22381 hybrid_clock.cc:648] HybridClock initialized: now 1786412638341753 us; error 58 us; skew 500 ppm
I20260811 01:43:58.342375 22381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:43:58 dist-test-slave-9gcw krb5kdc[22368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412638, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:43:58.347800 22381 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:43:58.350239 22381 webserver.cc:460] Webserver started at http://127.21.214.126:35795/ using document root <none> and password file <none>
I20260811 01:43:58.351202 22381 fs_manager.cc:359] Metadata directory not provided
I20260811 01:43:58.351392 22381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:43:58.351754 22381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:43:58.356119 22381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "cde75d9f07b746479f5274563dd9e150"
format_stamp: "Formatted at 2026-08-11 01:43:58 on dist-test-slave-9gcw"
server_key: "8263c9caa6e424942fca6a071f8e87fc"
server_key_iv: "c6f4714144b80a62f128ce444a804c66"
server_key_version: "encryptionkey@0"
I20260811 01:43:58.357592 22381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "cde75d9f07b746479f5274563dd9e150"
format_stamp: "Formatted at 2026-08-11 01:43:58 on dist-test-slave-9gcw"
server_key: "8263c9caa6e424942fca6a071f8e87fc"
server_key_iv: "c6f4714144b80a62f128ce444a804c66"
server_key_version: "encryptionkey@0"
I20260811 01:43:58.363988 22381 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 01:43:58.368961 22397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:43:58.370518 22381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 01:43:58.370748 22381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "cde75d9f07b746479f5274563dd9e150"
format_stamp: "Formatted at 2026-08-11 01:43:58 on dist-test-slave-9gcw"
server_key: "8263c9caa6e424942fca6a071f8e87fc"
server_key_iv: "c6f4714144b80a62f128ce444a804c66"
server_key_version: "encryptionkey@0"
I20260811 01:43:58.371023 22381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:43:58.397605 22381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:43:58.403697 22381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:43:58.404152 22381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:43:58.443202 22381 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:36883
I20260811 01:43:58.443272 22456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:36883 every 8 connection(s)
I20260811 01:43:58.444870 22381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:43:58.445994 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 22381
I20260811 01:43:58.446344 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:43:58.449174 22361 external_mini_cluster.cc:1428] Setting key a849e3e08cce0ebe05e0402d35a4add6
I20260811 01:43:58.451108 22457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:43:58.467373 22457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150: Bootstrap starting.
I20260811 01:43:58.471702 22457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150: Neither blocks nor log segments found. Creating new log.
I20260811 01:43:58.473502 22457 log.cc:824] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150: Log is configured to *not* fsync() on all Append() calls
I20260811 01:43:58.478035 22457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150: No bootstrap required, opened a new log
Aug 11 01:43:58 dist-test-slave-9gcw krb5kdc[22368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:43:58.488593 22457 raft_consensus.cc:348] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde75d9f07b746479f5274563dd9e150" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36883 } }
I20260811 01:43:58.489387 22457 raft_consensus.cc:374] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:43:58.489580 22457 raft_consensus.cc:729] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cde75d9f07b746479f5274563dd9e150, State: Initialized, Role: FOLLOWER
I20260811 01:43:58.490794 22457 consensus_queue.cc:260] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde75d9f07b746479f5274563dd9e150" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36883 } }
I20260811 01:43:58.491350 22457 raft_consensus.cc:388] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:43:58.491660 22457 raft_consensus.cc:482] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:43:58.492074 22457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:43:58.495505 22457 raft_consensus.cc:504] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde75d9f07b746479f5274563dd9e150" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36883 } }
I20260811 01:43:58.496189 22457 leader_election.cc:302] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cde75d9f07b746479f5274563dd9e150; no voters:
I20260811 01:43:58.497257 22457 leader_election.cc:288] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:43:58.497484 22465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:43:58.500710 22457 sys_catalog.cc:564] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:43:58.501040 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:43:58.454100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41296 (local address 127.21.214.126:36883)
0811 01:43:58.457678 (+ 3578us) server_negotiation.cc:206] Beginning negotiation
0811 01:43:58.457698 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 01:43:58.457773 (+ 75us) server_negotiation.cc:407] Connection header received
0811 01:43:58.469522 (+ 11749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:43:58.469549 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:43:58.470026 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:43:58.470488 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:43:58.474522 (+ 4034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:43:58.476996 (+ 2474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:43:58.479536 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:43:58.480778 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:43:58.480991 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:43:58.488364 (+ 7373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:43:58.488378 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:43:58.488399 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:43:58.488454 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:43:58.494495 (+ 6041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:43:58.495487 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:43:58.495495 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:43:58.495501 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:43:58.495794 (+ 293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:43:58.496401 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:43:58.496409 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:43:58.496413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:43:58.497253 (+ 840us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:43:58.497693 (+ 440us) server_negotiation.cc:1035] Waiting for connection context
0811 01:43:58.499070 (+ 1377us) server_negotiation.cc:299] Negotiation successful
0811 01:43:58.499524 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3268,"thread_start_us":2851,"threads_started":1}
I20260811 01:43:58.503675 22465 raft_consensus.cc:686] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150 [term 1 LEADER]: Becoming Leader. State: Replica: cde75d9f07b746479f5274563dd9e150, State: Running, Role: LEADER
I20260811 01:43:58.504720 22465 consensus_queue.cc:237] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde75d9f07b746479f5274563dd9e150" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36883 } }
I20260811 01:43:58.511477 22467 sys_catalog.cc:455] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cde75d9f07b746479f5274563dd9e150. Latest consensus state: current_term: 1 leader_uuid: "cde75d9f07b746479f5274563dd9e150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde75d9f07b746479f5274563dd9e150" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36883 } } }
I20260811 01:43:58.513602 22467 sys_catalog.cc:458] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150 [sys.catalog]: This master's current role is: LEADER
I20260811 01:43:58.511629 22466 sys_catalog.cc:455] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cde75d9f07b746479f5274563dd9e150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde75d9f07b746479f5274563dd9e150" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36883 } } }
I20260811 01:43:58.514868 22466 sys_catalog.cc:458] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150 [sys.catalog]: This master's current role is: LEADER
I20260811 01:43:58.517928 22473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:43:58.527781 22473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:43:58.541355 22473 catalog_manager.cc:1347] Generated new cluster ID: 6153336ad68b4ddd80f6ef3bbba08f2a
I20260811 01:43:58.541531 22473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:43:58.562621 22473 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:43:58.564185 22473 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:43:58.576793 22473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cde75d9f07b746479f5274563dd9e150: Generated new TSK 0
I20260811 01:43:58.577945 22473 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
I20260811 01:43:58.646138 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:36883
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:43:59.285565 22486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:43:59.286062 22486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:43:59.286352 22486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:43:59.296998 22486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:43:59.297333 22486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:43:59.297667 22486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:43:59.310848 22486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:36883
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:43:59.312232 22486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:43:59.313779 22486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:43:59.331203 22491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:43:59.331087 22494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:43:59.331813 22492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:43:59.333439 22486 server_base.cc:1034] running on GCE node
I20260811 01:43:59.334267 22486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:43:59.335457 22486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:43:59.336726 22486 hybrid_clock.cc:648] HybridClock initialized: now 1786412639336672 us; error 57 us; skew 500 ppm
I20260811 01:43:59.337131 22486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:43:59 dist-test-slave-9gcw krb5kdc[22368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412639, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:43:59.341495 22486 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:43:59.343472 22486 webserver.cc:460] Webserver started at http://127.21.214.65:43399/ using document root <none> and password file <none>
I20260811 01:43:59.344528 22486 fs_manager.cc:359] Metadata directory not provided
I20260811 01:43:59.344791 22486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:43:59.345410 22486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:43:59.349370 22486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "b8311619b40440d5b6b1a78ad6bca5c3"
format_stamp: "Formatted at 2026-08-11 01:43:59 on dist-test-slave-9gcw"
server_key: "ebde81c46c42f05f27b397df0bb43223"
server_key_iv: "e92955dd24713f226ac1ac55131f54d6"
server_key_version: "encryptionkey@0"
I20260811 01:43:59.350436 22486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "b8311619b40440d5b6b1a78ad6bca5c3"
format_stamp: "Formatted at 2026-08-11 01:43:59 on dist-test-slave-9gcw"
server_key: "ebde81c46c42f05f27b397df0bb43223"
server_key_iv: "e92955dd24713f226ac1ac55131f54d6"
server_key_version: "encryptionkey@0"
I20260811 01:43:59.358249 22486 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 01:43:59.362919 22501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:43:59.364387 22486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.005s
I20260811 01:43:59.364602 22486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "b8311619b40440d5b6b1a78ad6bca5c3"
format_stamp: "Formatted at 2026-08-11 01:43:59 on dist-test-slave-9gcw"
server_key: "ebde81c46c42f05f27b397df0bb43223"
server_key_iv: "e92955dd24713f226ac1ac55131f54d6"
server_key_version: "encryptionkey@0"
I20260811 01:43:59.364912 22486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:43:59.391033 22486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:43:59.397397 22486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:43:59.397851 22486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:43:59.399209 22486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:43:59.401105 22486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:43:59.401294 22486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 01:43:59.401499 22486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:43:59.401623 22486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:43:59.483340 22486 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:36203
I20260811 01:43:59.483491 22630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:36203 every 8 connection(s)
I20260811 01:43:59.484983 22486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:43:59.488281 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 22486
I20260811 01:43:59.488528 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:43:59.489660 22361 external_mini_cluster.cc:1428] Setting key c1f4abee4668da750d99bdf5219e1809
Aug 11 01:43:59 dist-test-slave-9gcw krb5kdc[22368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412639, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.214.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.66@KRBTEST.COM" created.
I20260811 01:43:59.516191 22633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:43:59.489062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:44079 (local address 127.21.214.126:36883)
0811 01:43:59.490282 (+ 1220us) server_negotiation.cc:206] Beginning negotiation
0811 01:43:59.490292 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:43:59.491138 (+ 846us) server_negotiation.cc:407] Connection header received
0811 01:43:59.495299 (+ 4161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:43:59.495307 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:43:59.495409 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:43:59.495513 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:43:59.498805 (+ 3292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:43:59.500292 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:43:59.502613 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:43:59.503303 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:43:59.503440 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:43:59.508366 (+ 4926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:43:59.508372 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:43:59.508375 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:43:59.508413 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:43:59.512071 (+ 3658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:43:59.512954 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:43:59.512958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:43:59.512960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:43:59.513244 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:43:59.513750 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:43:59.513755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:43:59.513759 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:43:59.514140 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:43:59.514317 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 01:43:59.515109 (+ 792us) server_negotiation.cc:299] Negotiation successful
0811 01:43:59.515426 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":982,"thread_start_us":759,"threads_started":1}
I20260811 01:43:59.517431 22634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:43:59.489564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36883 (local address 127.21.214.65:44079)
0811 01:43:59.490952 (+ 1388us) negotiation.cc:107] Waiting for socket to connect
0811 01:43:59.491006 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 01:43:59.494806 (+ 3800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:43:59.495909 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:43:59.495925 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:43:59.497283 (+ 1358us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:43:59.498546 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:43:59.498578 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:43:59.500478 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:43:59.500486 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:43:59.502427 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:43:59.502441 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:43:59.503605 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:43:59.503615 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:43:59.503886 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:43:59.504654 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:43:59.504678 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:43:59.507964 (+ 3286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:43:59.512231 (+ 4267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:43:59.512241 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:43:59.512260 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:43:59.512810 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:43:59.513377 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:43:59.513382 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:43:59.513386 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:43:59.513619 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:43:59.514283 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:43:59.514291 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:43:59.514715 (+ 424us) client_negotiation.cc:769] Sending connection context
0811 01:43:59.515011 (+ 296us) client_negotiation.cc:240] Negotiation successful
0811 01:43:59.515848 (+ 837us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1058,"thread_start_us":627,"threads_started":1}
I20260811 01:43:59.519527 22631 heartbeater.cc:347] Connected to a master server at 127.21.214.126:36883
I20260811 01:43:59.519908 22631 heartbeater.cc:464] Registering TS with master...
I20260811 01:43:59.520846 22631 heartbeater.cc:511] Master 127.21.214.126:36883 requested a full tablet report, sending...
I20260811 01:43:59.524048 22415 ts_manager.cc:194] Registered new tserver with Master: b8311619b40440d5b6b1a78ad6bca5c3 (127.21.214.65:36203)
I20260811 01:43:59.525869 22415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:44079
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
I20260811 01:43:59.543131 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:36883
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:00.131656 22638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:00.132054 22638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:00.132237 22638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:00.142899 22638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:00.143106 22638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:00.143257 22638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:44:00.155584 22638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:36883
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:00.157135 22638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:00.158640 22638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:00.179143 22644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:00.179059 22643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:00.179203 22646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:00.186512 22638 server_base.cc:1034] running on GCE node
I20260811 01:44:00.187412 22638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:00.188740 22638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:00.190042 22638 hybrid_clock.cc:648] HybridClock initialized: now 1786412640189977 us; error 70 us; skew 500 ppm
I20260811 01:44:00.190552 22638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:00 dist-test-slave-9gcw krb5kdc[22368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412640, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:00.195013 22638 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:44:00.197610 22638 webserver.cc:460] Webserver started at http://127.21.214.66:38755/ using document root <none> and password file <none>
I20260811 01:44:00.198519 22638 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:00.198709 22638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:00.199206 22638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:00.202873 22638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "b466eb398b3a43d39e5ab8ce2440457a"
format_stamp: "Formatted at 2026-08-11 01:44:00 on dist-test-slave-9gcw"
server_key: "4b2ebdad4a89201a1edcb86cb9337517"
server_key_iv: "b343ab954e6a3f8c868abf9ee11d0803"
server_key_version: "encryptionkey@0"
I20260811 01:44:00.203846 22638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "b466eb398b3a43d39e5ab8ce2440457a"
format_stamp: "Formatted at 2026-08-11 01:44:00 on dist-test-slave-9gcw"
server_key: "4b2ebdad4a89201a1edcb86cb9337517"
server_key_iv: "b343ab954e6a3f8c868abf9ee11d0803"
server_key_version: "encryptionkey@0"
I20260811 01:44:00.210803 22638 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.007s
I20260811 01:44:00.216019 22653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:00.217855 22638 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 01:44:00.218091 22638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "b466eb398b3a43d39e5ab8ce2440457a"
format_stamp: "Formatted at 2026-08-11 01:44:00 on dist-test-slave-9gcw"
server_key: "4b2ebdad4a89201a1edcb86cb9337517"
server_key_iv: "b343ab954e6a3f8c868abf9ee11d0803"
server_key_version: "encryptionkey@0"
I20260811 01:44:00.218327 22638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:00.259202 22638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:00.265055 22638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:00.265442 22638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:00.266744 22638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:00.268664 22638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:00.268800 22638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:00.268949 22638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:00.269016 22638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:00.344762 22638 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:45687
I20260811 01:44:00.344856 22782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:45687 every 8 connection(s)
I20260811 01:44:00.346377 22638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:44:00.354235 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 22638
I20260811 01:44:00.354525 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:44:00.355156 22361 external_mini_cluster.cc:1428] Setting key 6104978760a30a3034f6924693195f3d
Aug 11 01:44:00 dist-test-slave-9gcw krb5kdc[22368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412640, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.214.67@KRBTEST.COM; defaulting to no policy
I20260811 01:44:00.377440 22785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:00.350391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:43067 (local address 127.21.214.126:36883)
0811 01:44:00.351251 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:00.351256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:00.352264 (+ 1008us) server_negotiation.cc:407] Connection header received
0811 01:44:00.355743 (+ 3479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:00.355748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:00.355825 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:00.355919 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:00.358505 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:00.359356 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:00.362324 (+ 2968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:00.362906 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:00.363013 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:00.368365 (+ 5352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:00.368371 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:00.368374 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:00.368412 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:00.373009 (+ 4597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:00.374050 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:00.374059 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:00.374063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:00.374327 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:00.375118 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:00.375122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:00.375124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:00.375507 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:00.375697 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:00.376629 (+ 932us) server_negotiation.cc:299] Negotiation successful
0811 01:44:00.376897 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":637,"threads_started":1}
I20260811 01:44:00.379098 22786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:00.350923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36883 (local address 127.21.214.66:43067)
0811 01:44:00.352052 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:00.352101 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:00.355267 (+ 3166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:00.356220 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:00.356252 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:00.357195 (+ 943us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:00.358271 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:00.358336 (+ 65us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:00.359582 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:00.359589 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:00.362161 (+ 2572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:00.362181 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:00.363206 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:00.363216 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:00.363659 (+ 443us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:00.364657 (+ 998us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:00.364702 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:00.368027 (+ 3325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:00.373222 (+ 5195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:00.373234 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:00.373256 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:00.373855 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:00.374467 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:00.374473 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:00.374477 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:00.374968 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:00.375670 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:00.375678 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:00.376204 (+ 526us) client_negotiation.cc:769] Sending connection context
0811 01:44:00.376498 (+ 294us) client_negotiation.cc:240] Negotiation successful
0811 01:44:00.377238 (+ 740us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"thread_start_us":596,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.67@KRBTEST.COM" created.
I20260811 01:44:00.380986 22783 heartbeater.cc:347] Connected to a master server at 127.21.214.126:36883
I20260811 01:44:00.381397 22783 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:00.382601 22783 heartbeater.cc:511] Master 127.21.214.126:36883 requested a full tablet report, sending...
I20260811 01:44:00.384744 22415 ts_manager.cc:194] Registered new tserver with Master: b466eb398b3a43d39e5ab8ce2440457a (127.21.214.66:45687)
I20260811 01:44:00.386202 22415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:43067
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
I20260811 01:44:00.409740 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:36883
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:44:00.529498 22631 heartbeater.cc:503] Master 127.21.214.126:36883 was elected leader, sending a full tablet report...
W20260811 01:44:00.977936 22789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:00.978394 22789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:00.978624 22789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:00.988322 22789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:00.988559 22789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:00.988842 22789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:44:01.001578 22789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:36883
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:01.003234 22789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:01.004949 22789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:01.023000 22795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:01.023607 22796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:01.023864 22798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:01.026211 22789 server_base.cc:1034] running on GCE node
I20260811 01:44:01.027455 22789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:01.029233 22789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:01.030504 22789 hybrid_clock.cc:648] HybridClock initialized: now 1786412641030424 us; error 70 us; skew 500 ppm
I20260811 01:44:01.031123 22789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:01 dist-test-slave-9gcw krb5kdc[22368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412641, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:01.038664 22789 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:44:01.042302 22789 webserver.cc:460] Webserver started at http://127.21.214.67:44531/ using document root <none> and password file <none>
I20260811 01:44:01.043764 22789 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:01.044011 22789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:01.044636 22789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:01.050515 22789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "dc290520be2f4f6b8f47b3e967ee2c76"
format_stamp: "Formatted at 2026-08-11 01:44:01 on dist-test-slave-9gcw"
server_key: "2120d0888ebea6546d6e342e072a3c6d"
server_key_iv: "44333ceba29e12125a23a8581ba91d0c"
server_key_version: "encryptionkey@0"
I20260811 01:44:01.052030 22789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "dc290520be2f4f6b8f47b3e967ee2c76"
format_stamp: "Formatted at 2026-08-11 01:44:01 on dist-test-slave-9gcw"
server_key: "2120d0888ebea6546d6e342e072a3c6d"
server_key_iv: "44333ceba29e12125a23a8581ba91d0c"
server_key_version: "encryptionkey@0"
I20260811 01:44:01.061491 22789 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.008s sys 0.000s
I20260811 01:44:01.068041 22805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:01.070384 22789 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 01:44:01.070636 22789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "dc290520be2f4f6b8f47b3e967ee2c76"
format_stamp: "Formatted at 2026-08-11 01:44:01 on dist-test-slave-9gcw"
server_key: "2120d0888ebea6546d6e342e072a3c6d"
server_key_iv: "44333ceba29e12125a23a8581ba91d0c"
server_key_version: "encryptionkey@0"
I20260811 01:44:01.070940 22789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:01.107319 22789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:01.113332 22789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:01.113885 22789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:01.115288 22789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:01.117035 22789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:01.117187 22789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:01.117421 22789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:01.117583 22789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:01.192101 22789 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:40191
I20260811 01:44:01.192126 22932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:40191 every 8 connection(s)
I20260811 01:44:01.193998 22789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:44:01.198462 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 22789
I20260811 01:44:01.198839 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:44:01.199774 22361 external_mini_cluster.cc:1428] Setting key 0b0afaa2a4948c7e47441e042d001647
Aug 11 01:44:01 dist-test-slave-9gcw krb5kdc[22368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412641, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:01.225059 22935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:01.198464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:43321 (local address 127.21.214.126:36883)
0811 01:44:01.202138 (+ 3674us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:01.202145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:01.202167 (+ 22us) server_negotiation.cc:407] Connection header received
0811 01:44:01.204475 (+ 2308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:01.204481 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:01.204640 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:01.204791 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:01.207167 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:01.208174 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:01.210697 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:01.211286 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:01.211402 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:01.216673 (+ 5271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:01.216679 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:01.216682 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:01.216722 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:01.221064 (+ 4342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:01.222107 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:01.222113 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:01.222117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:01.222278 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:01.222990 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:01.222994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:01.222996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:01.223393 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:01.223536 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:01.224464 (+ 928us) server_negotiation.cc:299] Negotiation successful
0811 01:44:01.224646 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3534,"thread_start_us":3292,"threads_started":1}
I20260811 01:44:01.226388 22936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:01.199145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36883 (local address 127.21.214.67:43321)
0811 01:44:01.200742 (+ 1597us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:01.200800 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:01.204135 (+ 3335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:01.205095 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:01.205105 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:01.206132 (+ 1027us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:01.206975 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:01.206998 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:01.208366 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:01.208370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:01.210524 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:01.210539 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:01.211541 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:01.211547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:01.211859 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:01.212723 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:01.212750 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:01.216286 (+ 3536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:01.221286 (+ 5000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:01.221295 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:01.221312 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:01.221958 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:01.222386 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:01.222390 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:01.222392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:01.222861 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:01.223534 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:01.223541 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:01.224039 (+ 498us) client_negotiation.cc:769] Sending connection context
0811 01:44:01.224333 (+ 294us) client_negotiation.cc:240] Negotiation successful
0811 01:44:01.224837 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1209,"thread_start_us":815,"threads_started":1}
I20260811 01:44:01.228015 22933 heartbeater.cc:347] Connected to a master server at 127.21.214.126:36883
I20260811 01:44:01.228436 22933 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:01.229530 22933 heartbeater.cc:511] Master 127.21.214.126:36883 requested a full tablet report, sending...
I20260811 01:44:01.231420 22415 ts_manager.cc:194] Registered new tserver with Master: dc290520be2f4f6b8f47b3e967ee2c76 (127.21.214.67:40191)
I20260811 01:44:01.232275 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:44:01.232802 22415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:43321
Aug 11 01:44:01 dist-test-slave-9gcw krb5kdc[22368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412641, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 01:44:01 dist-test-slave-9gcw krb5kdc[22368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412641, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:44:01.293401 22942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:01.271530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36940 (local address 127.21.214.65:36203)
0811 01:44:01.272814 (+ 1284us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:01.272822 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:01.272850 (+ 28us) server_negotiation.cc:407] Connection header received
0811 01:44:01.272915 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:01.272922 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:01.273289 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:01.273485 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:01.276260 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:01.277856 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:01.279579 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:01.280280 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:01.280442 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:01.286167 (+ 5725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:01.286182 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:01.286200 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:01.286304 (+ 104us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:01.290036 (+ 3732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:01.290864 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:01.290872 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:01.290874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:01.291038 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:01.291492 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:01.291497 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:01.291501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:01.291875 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:01.292283 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:01.292643 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 01:44:01.292989 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":997,"thread_start_us":678,"threads_started":1}
W20260811 01:44:01.294737 22532 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36940
Aug 11 01:44:01 dist-test-slave-9gcw krb5kdc[22368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 01:44:01 dist-test-slave-9gcw krb5kdc[22368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:44:01.360538 22942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:01.333066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36944 (local address 127.21.214.65:36203)
0811 01:44:01.333336 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:01.333343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:01.333572 (+ 229us) server_negotiation.cc:407] Connection header received
0811 01:44:01.333732 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:01.333737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:01.333917 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:01.334016 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:01.335415 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:01.336335 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:01.337713 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:01.338376 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:01.338493 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:01.353596 (+ 15103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:01.353602 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:01.353606 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:01.353648 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:01.357390 (+ 3742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:01.358347 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:01.358351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:01.358354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:01.358602 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:01.359023 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:01.359026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:01.359028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:01.359363 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:01.359635 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:01.360007 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 01:44:01.360234 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 01:44:01.362867 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 22486
I20260811 01:44:01.380483 22942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:01.364146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36954 (local address 127.21.214.65:36203)
0811 01:44:01.364345 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:01.364349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:01.364691 (+ 342us) server_negotiation.cc:407] Connection header received
0811 01:44:01.364820 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:01.364824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:01.364888 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:01.364986 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:01.366777 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:01.367737 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:01.369459 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:01.370120 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:01.370238 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:01.373244 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:01.373251 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:01.373255 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:01.373288 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:01.376888 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:01.377943 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:01.377947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:01.377949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:01.378269 (+ 320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:01.378977 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:01.378982 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:01.378984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:01.379377 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:01.379590 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:01.379923 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 01:44:01.380108 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 01:44:01.381487 22626 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:01.389273 22783 heartbeater.cc:503] Master 127.21.214.126:36883 was elected leader, sending a full tablet report...
I20260811 01:44:01.470643 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 22486
I20260811 01:44:01.495784 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 22638
Aug 11 01:44:01 dist-test-slave-9gcw krb5kdc[22368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.66@KRBTEST.COM
I20260811 01:44:01.515368 22952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:01.497093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37496 (local address 127.21.214.66:45687)
0811 01:44:01.498189 (+ 1096us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:01.498197 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:01.498229 (+ 32us) server_negotiation.cc:407] Connection header received
0811 01:44:01.498299 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:01.498306 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:01.498576 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:01.498830 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:01.500125 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:01.501308 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:01.502732 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:01.503468 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:01.503587 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:01.508671 (+ 5084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:01.508684 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:01.508701 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:01.508740 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:01.512103 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:01.512924 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:01.512932 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:01.512935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:01.513069 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:01.513571 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:01.513579 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:01.513582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:01.513954 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:01.514332 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:01.514633 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 01:44:01.514949 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":850,"thread_start_us":626,"threads_started":1}
I20260811 01:44:01.516812 22778 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:01.611776 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 22638
I20260811 01:44:01.635946 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 22789
Aug 11 01:44:01 dist-test-slave-9gcw krb5kdc[22368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.67@KRBTEST.COM
I20260811 01:44:01.654631 22954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:01.636933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60284 (local address 127.21.214.67:40191)
0811 01:44:01.637560 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:01.637567 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:01.637599 (+ 32us) server_negotiation.cc:407] Connection header received
0811 01:44:01.637664 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:01.637670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:01.637906 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:01.638090 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:01.639483 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:01.640547 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:01.641974 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:01.642589 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:01.642764 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:01.647458 (+ 4694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:01.647472 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:01.647489 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:01.647534 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:01.650843 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:01.651590 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:01.651600 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:01.651604 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:01.651799 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:01.652324 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:01.652329 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:01.652333 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:01.652977 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:01.653496 (+ 519us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:01.653854 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 01:44:01.654150 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":313,"threads_started":1}
I20260811 01:44:01.656400 22928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:01.744587 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 22789
I20260811 01:44:01.766422 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 22381
I20260811 01:44:01.768014 22451 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:01.877365 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 22381
2026-08-11T01:44:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5658 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:44:01 dist-test-slave-9gcw krb5kdc[22959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:44:01 dist-test-slave-9gcw krb5kdc[22959](info): set up 2 sockets
Aug 11 01:44:01 dist-test-slave-9gcw krb5kdc[22959](info): commencing operation
krb5kdc: starting...
W20260811 01:44:03.078315 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:44:03 dist-test-slave-9gcw krb5kdc[22959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:44:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:44:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:44:03.297729 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:42079
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:33485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:42079
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:03.655625 22973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:03.655995 22973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:03.656177 22973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:03.665951 22973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:44:03.666095 22973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:03.666198 22973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:03.666294 22973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:44:03.666395 22973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:44:03.678185 22973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:33485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:42079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:42079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:03.679699 22973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:03.681268 22973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:03.694738 22980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:03.694710 22983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:03.694769 22979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:03.696089 22973 server_base.cc:1034] running on GCE node
I20260811 01:44:03.696949 22973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:03.698652 22973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:03.699914 22973 hybrid_clock.cc:648] HybridClock initialized: now 1786412643699871 us; error 51 us; skew 500 ppm
I20260811 01:44:03.700311 22973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:03 dist-test-slave-9gcw krb5kdc[22959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412643, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:03.704097 22973 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:44:03.705860 22973 webserver.cc:460] Webserver started at http://127.21.214.126:35827/ using document root <none> and password file <none>
I20260811 01:44:03.706614 22973 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:03.706766 22973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:03.707126 22973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:03.710611 22973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "00b227bb20a848b595ac68a3f772999b"
format_stamp: "Formatted at 2026-08-11 01:44:03 on dist-test-slave-9gcw"
server_key: "1c85cd9c16bf8f74acbbfa84c893bc4f"
server_key_iv: "15f3a272af978d0541398013e8d4da36"
server_key_version: "encryptionkey@0"
I20260811 01:44:03.711469 22973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "00b227bb20a848b595ac68a3f772999b"
format_stamp: "Formatted at 2026-08-11 01:44:03 on dist-test-slave-9gcw"
server_key: "1c85cd9c16bf8f74acbbfa84c893bc4f"
server_key_iv: "15f3a272af978d0541398013e8d4da36"
server_key_version: "encryptionkey@0"
I20260811 01:44:03.717507 22973 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 01:44:03.721438 22990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:03.722998 22973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:44:03.723193 22973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "00b227bb20a848b595ac68a3f772999b"
format_stamp: "Formatted at 2026-08-11 01:44:03 on dist-test-slave-9gcw"
server_key: "1c85cd9c16bf8f74acbbfa84c893bc4f"
server_key_iv: "15f3a272af978d0541398013e8d4da36"
server_key_version: "encryptionkey@0"
I20260811 01:44:03.723428 22973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:03.735313 22973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:03.739835 22973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:03.740171 22973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:03.760237 22973 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:42079
I20260811 01:44:03.760223 23044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:42079 every 8 connection(s)
I20260811 01:44:03.761822 22973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:44:03.765771 23046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:03.769321 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 22973
I20260811 01:44:03.769567 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:44:03.770434 22361 external_mini_cluster.cc:1428] Setting key 36afe7b63c95a55e8691d0aee2b99665
I20260811 01:44:03.779939 23046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b: Bootstrap starting.
I20260811 01:44:03.783339 23046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:03.784693 23046 log.cc:824] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b: Log is configured to *not* fsync() on all Append() calls
Aug 11 01:44:03 dist-test-slave-9gcw krb5kdc[22959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:03.788127 23046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b: No bootstrap required, opened a new log
I20260811 01:44:03.795696 23050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:03.772537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35146 (local address 127.21.214.126:42079)
0811 01:44:03.773396 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:03.773412 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:03.773466 (+ 54us) server_negotiation.cc:407] Connection header received
0811 01:44:03.775486 (+ 2020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:03.775522 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:03.776137 (+ 615us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:03.776614 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:03.778649 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:03.780418 (+ 1769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:03.781955 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:03.782731 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:03.782962 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:03.787458 (+ 4496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:03.787467 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:03.787483 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:03.787533 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:03.790980 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:03.791871 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:03.791879 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:03.791886 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:03.792055 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:03.792514 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:03.792518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:03.792520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:03.793128 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:03.793540 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:03.794094 (+ 554us) server_negotiation.cc:299] Negotiation successful
0811 01:44:03.794579 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":348,"threads_started":1}
I20260811 01:44:03.796339 23046 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b227bb20a848b595ac68a3f772999b" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42079 } }
I20260811 01:44:03.796759 23046 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:03.796890 23046 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00b227bb20a848b595ac68a3f772999b, State: Initialized, Role: FOLLOWER
I20260811 01:44:03.797652 23046 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b227bb20a848b595ac68a3f772999b" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42079 } }
I20260811 01:44:03.797966 23046 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:44:03.798118 23046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:44:03.798309 23046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:03.800923 23046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b227bb20a848b595ac68a3f772999b" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42079 } }
I20260811 01:44:03.801498 23046 leader_election.cc:302] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00b227bb20a848b595ac68a3f772999b; no voters:
I20260811 01:44:03.802460 23046 leader_election.cc:288] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:44:03.802536 23052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:03.803458 23052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b [term 1 LEADER]: Becoming Leader. State: Replica: 00b227bb20a848b595ac68a3f772999b, State: Running, Role: LEADER
I20260811 01:44:03.804067 23052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b227bb20a848b595ac68a3f772999b" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42079 } }
I20260811 01:44:03.805019 23046 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:44:03.807377 23053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00b227bb20a848b595ac68a3f772999b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b227bb20a848b595ac68a3f772999b" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42079 } } }
I20260811 01:44:03.807876 23053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:03.810180 23054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00b227bb20a848b595ac68a3f772999b. Latest consensus state: current_term: 1 leader_uuid: "00b227bb20a848b595ac68a3f772999b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b227bb20a848b595ac68a3f772999b" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42079 } } }
I20260811 01:44:03.810526 23054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:03.814273 23061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:44:03.819784 23061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:44:03.828642 23061 catalog_manager.cc:1347] Generated new cluster ID: 3fadbeae4b0147bfa43f8e860bc65786
I20260811 01:44:03.828778 23061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:44:03.868619 23061 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:44:03.870257 23061 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:44:03.881033 23061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00b227bb20a848b595ac68a3f772999b: Generated new TSK 0
I20260811 01:44:03.882089 23061 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
I20260811 01:44:03.941816 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42079
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:33485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:04.312565 23075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:04.312879 23075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:04.313068 23075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:04.321533 23075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:04.321686 23075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:04.321909 23075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:44:04.334292 23075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:33485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42079
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:04.335727 23075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:04.337286 23075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:04.352483 23082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:04.352592 23081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:04.352813 23084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:04.353891 23075 server_base.cc:1034] running on GCE node
I20260811 01:44:04.354594 23075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:04.355755 23075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:04.356993 23075 hybrid_clock.cc:648] HybridClock initialized: now 1786412644356945 us; error 51 us; skew 500 ppm
I20260811 01:44:04.357393 23075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:04 dist-test-slave-9gcw krb5kdc[22959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412644, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:04.360738 23075 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:44:04.362553 23075 webserver.cc:460] Webserver started at http://127.21.214.65:46883/ using document root <none> and password file <none>
I20260811 01:44:04.363336 23075 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:04.363493 23075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:04.363870 23075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:04.367224 23075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "82119c8dfc5a429eab294d56af4288ff"
format_stamp: "Formatted at 2026-08-11 01:44:04 on dist-test-slave-9gcw"
server_key: "72a2c109a3a13d313e6eaf7dc181946b"
server_key_iv: "c2cfd6d3d855c869b28c3e475fa8f53b"
server_key_version: "encryptionkey@0"
I20260811 01:44:04.368206 23075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "82119c8dfc5a429eab294d56af4288ff"
format_stamp: "Formatted at 2026-08-11 01:44:04 on dist-test-slave-9gcw"
server_key: "72a2c109a3a13d313e6eaf7dc181946b"
server_key_iv: "c2cfd6d3d855c869b28c3e475fa8f53b"
server_key_version: "encryptionkey@0"
I20260811 01:44:04.374118 23075 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 01:44:04.377965 23091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:04.379401 23075 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:44:04.379609 23075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "82119c8dfc5a429eab294d56af4288ff"
format_stamp: "Formatted at 2026-08-11 01:44:04 on dist-test-slave-9gcw"
server_key: "72a2c109a3a13d313e6eaf7dc181946b"
server_key_iv: "c2cfd6d3d855c869b28c3e475fa8f53b"
server_key_version: "encryptionkey@0"
I20260811 01:44:04.379920 23075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:04.399964 23075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:04.404438 23075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:04.404702 23075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:04.405967 23075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:04.407631 23075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:04.407728 23075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:04.407889 23075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:04.407959 23075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:04.445534 23075 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:41639
I20260811 01:44:04.445705 23222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:41639 every 8 connection(s)
I20260811 01:44:04.446933 23075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:44:04.455019 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 23075
I20260811 01:44:04.455284 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:44:04.456151 22361 external_mini_cluster.cc:1428] Setting key 5888eb23898b171b14448557ebabbe41
Aug 11 01:44:04 dist-test-slave-9gcw krb5kdc[22959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412644, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:04.470193 23225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:04.449849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:52633 (local address 127.21.214.126:42079)
0811 01:44:04.450334 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:04.450339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:04.451385 (+ 1046us) server_negotiation.cc:407] Connection header received
0811 01:44:04.453357 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:04.453361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:04.453441 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:04.453544 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:04.455212 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:04.456459 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:04.458482 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:04.459107 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:04.459279 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:04.463221 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:04.463230 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:04.463238 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:04.463286 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:04.466395 (+ 3109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:04.467506 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:04.467510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:04.467512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:04.467643 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:04.468181 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:04.468184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:04.468186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:04.468511 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:04.468627 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:04.469732 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0811 01:44:04.469924 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":241,"threads_started":1}
I20260811 01:44:04.471907 23226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:04.450446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:42079 (local address 127.21.214.65:52633)
0811 01:44:04.451213 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:04.451253 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:04.453001 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:04.453804 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:04.453813 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:04.454393 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:04.454998 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:04.455017 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:04.456664 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:04.456669 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:04.458338 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:04.458351 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:04.459349 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:04.459355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:04.459563 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:04.460357 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:04.460386 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:04.462973 (+ 2587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:04.466567 (+ 3594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:04.466580 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:04.466602 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:04.467358 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:04.467761 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:04.467768 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:04.467775 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:04.468048 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:04.468649 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:04.468660 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:04.469271 (+ 611us) client_negotiation.cc:769] Sending connection context
0811 01:44:04.469635 (+ 364us) client_negotiation.cc:240] Negotiation successful
0811 01:44:04.470276 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":302,"threads_started":1}
I20260811 01:44:04.473821 23223 heartbeater.cc:347] Connected to a master server at 127.21.214.126:42079
I20260811 01:44:04.474146 23223 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:04.474958 23223 heartbeater.cc:511] Master 127.21.214.126:42079 requested a full tablet report, sending...
I20260811 01:44:04.477370 23007 ts_manager.cc:194] Registered new tserver with Master: 82119c8dfc5a429eab294d56af4288ff (127.21.214.65:41639)
WARNING: no policy specified for kudu/127.21.214.66@KRBTEST.COM; defaulting to no policy
I20260811 01:44:04.478901 23007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:52633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
I20260811 01:44:04.508706 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42079
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:33485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:04.857539 23229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:04.857892 23229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:04.858130 23229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:04.865751 23229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:04.865947 23229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:04.866184 23229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:44:04.877133 23229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:33485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42079
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:04.878746 23229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:04.880259 23229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:04.895867 23237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:04.895961 23239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:04.895973 23236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:04.897089 23229 server_base.cc:1034] running on GCE node
I20260811 01:44:04.897912 23229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:04.899024 23229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:04.900285 23229 hybrid_clock.cc:648] HybridClock initialized: now 1786412644900237 us; error 38 us; skew 500 ppm
I20260811 01:44:04.900668 23229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:04 dist-test-slave-9gcw krb5kdc[22959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412644, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:04.904253 23229 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:44:04.906149 23229 webserver.cc:460] Webserver started at http://127.21.214.66:36691/ using document root <none> and password file <none>
I20260811 01:44:04.906960 23229 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:04.907104 23229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:04.907400 23229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:04.910674 23229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "8d7c03ac704b4cfab6c12b3274db129b"
format_stamp: "Formatted at 2026-08-11 01:44:04 on dist-test-slave-9gcw"
server_key: "857384c618f555b2bd3d5b48cffb090e"
server_key_iv: "c2ad1d77c8b07c24473bca67738e9ce5"
server_key_version: "encryptionkey@0"
I20260811 01:44:04.911448 23229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "8d7c03ac704b4cfab6c12b3274db129b"
format_stamp: "Formatted at 2026-08-11 01:44:04 on dist-test-slave-9gcw"
server_key: "857384c618f555b2bd3d5b48cffb090e"
server_key_iv: "c2ad1d77c8b07c24473bca67738e9ce5"
server_key_version: "encryptionkey@0"
I20260811 01:44:04.917050 23229 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260811 01:44:04.920853 23246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:04.922428 23229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:44:04.922603 23229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "8d7c03ac704b4cfab6c12b3274db129b"
format_stamp: "Formatted at 2026-08-11 01:44:04 on dist-test-slave-9gcw"
server_key: "857384c618f555b2bd3d5b48cffb090e"
server_key_iv: "c2ad1d77c8b07c24473bca67738e9ce5"
server_key_version: "encryptionkey@0"
I20260811 01:44:04.922853 23229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:04.972543 23229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:04.977561 23229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:04.977890 23229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:04.978930 23229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:04.980341 23229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:04.980448 23229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:04.980605 23229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:04.980799 23229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:05.015828 23229 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:35121
I20260811 01:44:05.015879 23378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:35121 every 8 connection(s)
I20260811 01:44:05.017297 23229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:44:05.020676 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 23229
I20260811 01:44:05.020891 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:44:05.021567 22361 external_mini_cluster.cc:1428] Setting key af59aeec32df7f9897177162e5d12324
Aug 11 01:44:05 dist-test-slave-9gcw krb5kdc[22959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412644, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:05.041281 23381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:05.020131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:38995 (local address 127.21.214.126:42079)
0811 01:44:05.020712 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:05.020719 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:05.021939 (+ 1220us) server_negotiation.cc:407] Connection header received
0811 01:44:05.024278 (+ 2339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:05.024282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:05.024364 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:05.024474 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:05.026738 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.027574 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.030329 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.030770 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.030911 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:05.034922 (+ 4011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:05.034927 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:05.034930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:05.034966 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:05.037844 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:05.038710 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:05.038715 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:05.038718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:05.038852 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:05.039352 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:05.039356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:05.039358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:05.039668 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:05.039837 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:05.040604 (+ 767us) server_negotiation.cc:299] Negotiation successful
0811 01:44:05.040857 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":329,"threads_started":1}
WARNING: no policy specified for kudu/127.21.214.67@KRBTEST.COM; defaulting to no policy
I20260811 01:44:05.042447 23383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:05.020782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:42079 (local address 127.21.214.66:38995)
0811 01:44:05.021735 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:05.021801 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:05.023942 (+ 2141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:05.024764 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:05.024778 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:05.025635 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:05.026518 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:05.026547 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.027773 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.027781 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:05.030131 (+ 2350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:05.030149 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.031024 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.031032 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:05.031283 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:05.032040 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:05.032064 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:05.034614 (+ 2550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:05.038026 (+ 3412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:05.038037 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:05.038065 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:05.038547 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:05.038979 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:05.038984 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:05.038993 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:05.039215 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:05.039813 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:05.039822 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:05.040226 (+ 404us) client_negotiation.cc:769] Sending connection context
0811 01:44:05.040517 (+ 291us) client_negotiation.cc:240] Negotiation successful
0811 01:44:05.041106 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":394,"threads_started":1}
I20260811 01:44:05.044183 23379 heartbeater.cc:347] Connected to a master server at 127.21.214.126:42079
I20260811 01:44:05.044481 23379 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:05.045338 23379 heartbeater.cc:511] Master 127.21.214.126:42079 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.67@KRBTEST.COM" created.
I20260811 01:44:05.047044 23007 ts_manager.cc:194] Registered new tserver with Master: 8d7c03ac704b4cfab6c12b3274db129b (127.21.214.66:35121)
I20260811 01:44:05.047935 23007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:38995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
I20260811 01:44:05.074177 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42079
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:33485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:05.450783 23386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:05.451054 23386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:05.451165 23386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:05.458814 23386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:05.458950 23386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:05.459071 23386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:44:05.469203 23386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:33485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42079
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:05.470525 23386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:05.471886 23386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 01:44:05.482316 23223 heartbeater.cc:503] Master 127.21.214.126:42079 was elected leader, sending a full tablet report...
W20260811 01:44:05.486760 23392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:05.486726 23394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:05.487082 23397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:05.487725 23386 server_base.cc:1034] running on GCE node
I20260811 01:44:05.488438 23386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:05.489547 23386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:05.490800 23386 hybrid_clock.cc:648] HybridClock initialized: now 1786412645490693 us; error 57 us; skew 500 ppm
I20260811 01:44:05.491155 23386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:05 dist-test-slave-9gcw krb5kdc[22959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412645, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:05.494504 23386 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:44:05.496057 23386 webserver.cc:460] Webserver started at http://127.21.214.67:46521/ using document root <none> and password file <none>
I20260811 01:44:05.496794 23386 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:05.496946 23386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:05.497370 23386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:05.500473 23386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "af7cbe6d37d14c34bd9a9a4f8f02b94e"
format_stamp: "Formatted at 2026-08-11 01:44:05 on dist-test-slave-9gcw"
server_key: "0931864a4a9010191aa07f41bbb4a41a"
server_key_iv: "6748a63593d4bdff34ceb73db1887eef"
server_key_version: "encryptionkey@0"
I20260811 01:44:05.501405 23386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "af7cbe6d37d14c34bd9a9a4f8f02b94e"
format_stamp: "Formatted at 2026-08-11 01:44:05 on dist-test-slave-9gcw"
server_key: "0931864a4a9010191aa07f41bbb4a41a"
server_key_iv: "6748a63593d4bdff34ceb73db1887eef"
server_key_version: "encryptionkey@0"
I20260811 01:44:05.507376 23386 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 01:44:05.511260 23405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:05.512687 23386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 01:44:05.512871 23386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "af7cbe6d37d14c34bd9a9a4f8f02b94e"
format_stamp: "Formatted at 2026-08-11 01:44:05 on dist-test-slave-9gcw"
server_key: "0931864a4a9010191aa07f41bbb4a41a"
server_key_iv: "6748a63593d4bdff34ceb73db1887eef"
server_key_version: "encryptionkey@0"
I20260811 01:44:05.513094 23386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:05.548077 23386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:05.552438 23386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:05.552745 23386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:05.553823 23386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:05.555197 23386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:05.555311 23386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:05.555490 23386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:05.555564 23386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:05.593304 23386 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:44449
I20260811 01:44:05.593369 23536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:44449 every 8 connection(s)
I20260811 01:44:05.594950 23386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:44:05.598069 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 23386
I20260811 01:44:05.598320 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:44:05.598886 22361 external_mini_cluster.cc:1428] Setting key 231bac6060ba3a33308a556b919e8e30
Aug 11 01:44:05 dist-test-slave-9gcw krb5kdc[22959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412645, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:05.620482 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:05.598455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:52499 (local address 127.21.214.126:42079)
0811 01:44:05.598916 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:05.598921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:05.600986 (+ 2065us) server_negotiation.cc:407] Connection header received
0811 01:44:05.603837 (+ 2851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:05.603841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:05.603913 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:05.604004 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:05.605862 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.607015 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.609123 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.609669 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.609780 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:05.613803 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:05.613808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:05.613812 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:05.613847 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:05.616891 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:05.617814 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:05.617823 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:05.617826 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:05.618013 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:05.618591 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:05.618594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:05.618596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:05.618941 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:05.619066 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:05.619879 (+ 813us) server_negotiation.cc:299] Negotiation successful
0811 01:44:05.620060 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":241,"threads_started":1}
I20260811 01:44:05.621716 23540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:05.599222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:42079 (local address 127.21.214.67:52499)
0811 01:44:05.600785 (+ 1563us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:05.600863 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:05.603574 (+ 2711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:05.604227 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:05.604235 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:05.604848 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:05.605669 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:05.605692 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.607156 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.607160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:05.608982 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:05.608995 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.609909 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.609914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:05.610140 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:05.610842 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:05.610877 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:05.613549 (+ 2672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:05.617096 (+ 3547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:05.617106 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:05.617127 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:05.617672 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:05.618157 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:05.618161 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:05.618163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:05.618468 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:05.619060 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:05.619066 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:05.619497 (+ 431us) client_negotiation.cc:769] Sending connection context
0811 01:44:05.619774 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 01:44:05.620232 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1196,"thread_start_us":911,"threads_started":1}
I20260811 01:44:05.623426 23537 heartbeater.cc:347] Connected to a master server at 127.21.214.126:42079
I20260811 01:44:05.623785 23537 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:05.624794 23537 heartbeater.cc:511] Master 127.21.214.126:42079 requested a full tablet report, sending...
I20260811 01:44:05.626919 23007 ts_manager.cc:194] Registered new tserver with Master: af7cbe6d37d14c34bd9a9a4f8f02b94e (127.21.214.67:44449)
I20260811 01:44:05.627955 23007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:52499
I20260811 01:44:05.629032 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:44:05.655035 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:05.638605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35160 (local address 127.21.214.126:42079)
0811 01:44:05.638831 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:05.638838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:05.639784 (+ 946us) server_negotiation.cc:407] Connection header received
0811 01:44:05.639953 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:05.639958 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:05.640036 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:05.640155 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:05.642761 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.643710 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.645113 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.645744 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.645853 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:05.648950 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:05.648954 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:05.648958 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:05.648992 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:05.652216 (+ 3224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:05.652867 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:05.652870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:05.652873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:05.653030 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:05.653502 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:05.653505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:05.653507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:05.653900 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:05.654125 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:05.654462 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 01:44:05.654640 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 01:44:05.662847 23006 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:44:05.674594 23006 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:44:05.708909 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:05.692010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55886 (local address 127.21.214.65:41639)
0811 01:44:05.694337 (+ 2327us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:05.694350 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:05.694387 (+ 37us) server_negotiation.cc:407] Connection header received
0811 01:44:05.694508 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:05.694520 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:05.694838 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:05.698056 (+ 3218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:05.698580 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.700738 (+ 2158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.705118 (+ 4380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.707398 (+ 2280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.707556 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:05.707859 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:05.708211 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 01:44:05.708430 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2097,"thread_start_us":1870,"threads_started":1}
I20260811 01:44:05.709643 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:05.691250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:41639 (local address 127.0.0.1:55886)
0811 01:44:05.692860 (+ 1610us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:05.692908 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:05.693186 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:05.696581 (+ 3395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:05.696591 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:05.696631 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:05.697692 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:05.697712 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.700911 (+ 3199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.700916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:05.704807 (+ 3891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:05.704824 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.707665 (+ 2841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.707672 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:05.707892 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:05.707944 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 01:44:05.708102 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 01:44:05.708379 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1203,"mutex_wait_us":1,"thread_start_us":879,"threads_started":1}
I20260811 01:44:05.709860 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:05.692282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34858 (local address 127.21.214.66:35121)
0811 01:44:05.695576 (+ 3294us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:05.695589 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:05.696933 (+ 1344us) server_negotiation.cc:407] Connection header received
0811 01:44:05.697059 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:05.697068 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:05.697470 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:05.697768 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:05.699204 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.700939 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.706341 (+ 5402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.708122 (+ 1781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.708324 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:05.708511 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:05.709283 (+ 772us) server_negotiation.cc:299] Negotiation successful
0811 01:44:05.709495 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3097,"thread_start_us":2957,"threads_started":1}
I20260811 01:44:05.710807 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:05.691716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:35121 (local address 127.0.0.1:34858)
0811 01:44:05.696573 (+ 4857us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:05.696593 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:05.696752 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:05.698056 (+ 1304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:05.698061 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:05.698087 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:05.698991 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:05.699006 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.701103 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.701113 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:05.706110 (+ 4997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:05.706130 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.708754 (+ 2624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.708762 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:05.709012 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:05.709058 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 01:44:05.710127 (+ 1069us) client_negotiation.cc:240] Negotiation successful
0811 01:44:05.710277 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4677,"thread_start_us":4314,"threads_started":1}
I20260811 01:44:05.714190 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:05.693244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:44449 (local address 127.0.0.1:55976)
0811 01:44:05.702417 (+ 9173us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:05.702451 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:05.702626 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:05.704752 (+ 2126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:05.704757 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:05.704786 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:05.706204 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:05.706218 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.708301 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.708306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:05.711223 (+ 2917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:05.711237 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.713438 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.713445 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:05.713618 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:05.713661 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 01:44:05.713737 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 01:44:05.713892 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8940,"thread_start_us":8538,"threads_started":1}
I20260811 01:44:05.714779 23560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:05.694678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55976 (local address 127.21.214.67:44449)
0811 01:44:05.704177 (+ 9499us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:05.704184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:05.704217 (+ 33us) server_negotiation.cc:407] Connection header received
0811 01:44:05.704298 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:05.704303 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:05.704497 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:05.704799 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:05.706533 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.708139 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.711399 (+ 3260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.713223 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.713373 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:05.713751 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:05.713852 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 01:44:05.714065 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9301,"thread_start_us":9084,"threads_started":1}
I20260811 01:44:05.715327 23308 tablet_service.cc:1467] Processing CreateTablet for tablet 656b5c06958243e7bdf5c2718dfc19e1 (DEFAULT_TABLE table=test-table [id=16f4bd7cd73547b98292c5d732117ff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:05.715286 23149 tablet_service.cc:1467] Processing CreateTablet for tablet 656b5c06958243e7bdf5c2718dfc19e1 (DEFAULT_TABLE table=test-table [id=16f4bd7cd73547b98292c5d732117ff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:05.717317 23149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 656b5c06958243e7bdf5c2718dfc19e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:05.718428 23308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 656b5c06958243e7bdf5c2718dfc19e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:05.719491 23465 tablet_service.cc:1467] Processing CreateTablet for tablet 656b5c06958243e7bdf5c2718dfc19e1 (DEFAULT_TABLE table=test-table [id=16f4bd7cd73547b98292c5d732117ff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:05.721458 23465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 656b5c06958243e7bdf5c2718dfc19e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:05.737303 23561 tablet_bootstrap.cc:492] T 656b5c06958243e7bdf5c2718dfc19e1 P 82119c8dfc5a429eab294d56af4288ff: Bootstrap starting.
I20260811 01:44:05.741448 23561 tablet_bootstrap.cc:654] T 656b5c06958243e7bdf5c2718dfc19e1 P 82119c8dfc5a429eab294d56af4288ff: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:05.743008 23562 tablet_bootstrap.cc:492] T 656b5c06958243e7bdf5c2718dfc19e1 P 8d7c03ac704b4cfab6c12b3274db129b: Bootstrap starting.
I20260811 01:44:05.743585 23561 log.cc:824] T 656b5c06958243e7bdf5c2718dfc19e1 P 82119c8dfc5a429eab294d56af4288ff: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:05.744766 23563 tablet_bootstrap.cc:492] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e: Bootstrap starting.
I20260811 01:44:05.750754 23562 tablet_bootstrap.cc:654] T 656b5c06958243e7bdf5c2718dfc19e1 P 8d7c03ac704b4cfab6c12b3274db129b: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:05.752400 23562 log.cc:824] T 656b5c06958243e7bdf5c2718dfc19e1 P 8d7c03ac704b4cfab6c12b3274db129b: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:05.752998 23563 tablet_bootstrap.cc:654] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:05.753513 23561 tablet_bootstrap.cc:492] T 656b5c06958243e7bdf5c2718dfc19e1 P 82119c8dfc5a429eab294d56af4288ff: No bootstrap required, opened a new log
I20260811 01:44:05.753917 23561 ts_tablet_manager.cc:1397] T 656b5c06958243e7bdf5c2718dfc19e1 P 82119c8dfc5a429eab294d56af4288ff: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.012s
I20260811 01:44:05.755236 23563 log.cc:824] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:05.756906 23562 tablet_bootstrap.cc:492] T 656b5c06958243e7bdf5c2718dfc19e1 P 8d7c03ac704b4cfab6c12b3274db129b: No bootstrap required, opened a new log
I20260811 01:44:05.757258 23562 ts_tablet_manager.cc:1397] T 656b5c06958243e7bdf5c2718dfc19e1 P 8d7c03ac704b4cfab6c12b3274db129b: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.003s
I20260811 01:44:05.758857 23563 tablet_bootstrap.cc:492] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e: No bootstrap required, opened a new log
I20260811 01:44:05.759189 23563 ts_tablet_manager.cc:1397] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20260811 01:44:05.769990 23561 raft_consensus.cc:348] T 656b5c06958243e7bdf5c2718dfc19e1 P 82119c8dfc5a429eab294d56af4288ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82119c8dfc5a429eab294d56af4288ff" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 41639 } } peers { permanent_uuid: "af7cbe6d37d14c34bd9a9a4f8f02b94e" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 44449 } } peers { permanent_uuid: "8d7c03ac704b4cfab6c12b3274db129b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 35121 } }
I20260811 01:44:05.770880 23561 raft_consensus.cc:374] T 656b5c06958243e7bdf5c2718dfc19e1 P 82119c8dfc5a429eab294d56af4288ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:05.771090 23561 raft_consensus.cc:729] T 656b5c06958243e7bdf5c2718dfc19e1 P 82119c8dfc5a429eab294d56af4288ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82119c8dfc5a429eab294d56af4288ff, State: Initialized, Role: FOLLOWER
I20260811 01:44:05.772084 23561 consensus_queue.cc:260] T 656b5c06958243e7bdf5c2718dfc19e1 P 82119c8dfc5a429eab294d56af4288ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82119c8dfc5a429eab294d56af4288ff" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 41639 } } peers { permanent_uuid: "af7cbe6d37d14c34bd9a9a4f8f02b94e" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 44449 } } peers { permanent_uuid: "8d7c03ac704b4cfab6c12b3274db129b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 35121 } }
I20260811 01:44:05.771956 23562 raft_consensus.cc:348] T 656b5c06958243e7bdf5c2718dfc19e1 P 8d7c03ac704b4cfab6c12b3274db129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82119c8dfc5a429eab294d56af4288ff" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 41639 } } peers { permanent_uuid: "af7cbe6d37d14c34bd9a9a4f8f02b94e" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 44449 } } peers { permanent_uuid: "8d7c03ac704b4cfab6c12b3274db129b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 35121 } }
I20260811 01:44:05.772607 23562 raft_consensus.cc:374] T 656b5c06958243e7bdf5c2718dfc19e1 P 8d7c03ac704b4cfab6c12b3274db129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:05.772799 23562 raft_consensus.cc:729] T 656b5c06958243e7bdf5c2718dfc19e1 P 8d7c03ac704b4cfab6c12b3274db129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d7c03ac704b4cfab6c12b3274db129b, State: Initialized, Role: FOLLOWER
I20260811 01:44:05.774025 23562 consensus_queue.cc:260] T 656b5c06958243e7bdf5c2718dfc19e1 P 8d7c03ac704b4cfab6c12b3274db129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82119c8dfc5a429eab294d56af4288ff" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 41639 } } peers { permanent_uuid: "af7cbe6d37d14c34bd9a9a4f8f02b94e" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 44449 } } peers { permanent_uuid: "8d7c03ac704b4cfab6c12b3274db129b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 35121 } }
I20260811 01:44:05.775207 23563 raft_consensus.cc:348] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82119c8dfc5a429eab294d56af4288ff" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 41639 } } peers { permanent_uuid: "af7cbe6d37d14c34bd9a9a4f8f02b94e" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 44449 } } peers { permanent_uuid: "8d7c03ac704b4cfab6c12b3274db129b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 35121 } }
I20260811 01:44:05.775858 23563 raft_consensus.cc:374] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:05.776073 23563 raft_consensus.cc:729] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af7cbe6d37d14c34bd9a9a4f8f02b94e, State: Initialized, Role: FOLLOWER
I20260811 01:44:05.776890 23563 consensus_queue.cc:260] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82119c8dfc5a429eab294d56af4288ff" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 41639 } } peers { permanent_uuid: "af7cbe6d37d14c34bd9a9a4f8f02b94e" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 44449 } } peers { permanent_uuid: "8d7c03ac704b4cfab6c12b3274db129b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 35121 } }
I20260811 01:44:05.781962 23379 heartbeater.cc:503] Master 127.21.214.126:42079 was elected leader, sending a full tablet report...
I20260811 01:44:05.783185 23562 ts_tablet_manager.cc:1428] T 656b5c06958243e7bdf5c2718dfc19e1 P 8d7c03ac704b4cfab6c12b3274db129b: Time spent starting tablet: real 0.026s user 0.015s sys 0.009s
I20260811 01:44:05.784440 23537 heartbeater.cc:503] Master 127.21.214.126:42079 was elected leader, sending a full tablet report...
I20260811 01:44:05.785705 23563 ts_tablet_manager.cc:1428] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e: Time spent starting tablet: real 0.026s user 0.020s sys 0.005s
I20260811 01:44:05.785784 23561 ts_tablet_manager.cc:1428] T 656b5c06958243e7bdf5c2718dfc19e1 P 82119c8dfc5a429eab294d56af4288ff: Time spent starting tablet: real 0.032s user 0.013s sys 0.012s
W20260811 01:44:05.847559 23538 tablet.cc:2369] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:44:05.909811 23570 raft_consensus.cc:482] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:44:05.910327 23570 raft_consensus.cc:504] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82119c8dfc5a429eab294d56af4288ff" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 41639 } } peers { permanent_uuid: "af7cbe6d37d14c34bd9a9a4f8f02b94e" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 44449 } } peers { permanent_uuid: "8d7c03ac704b4cfab6c12b3274db129b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 35121 } }
I20260811 01:44:05.912715 23570 leader_election.cc:288] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82119c8dfc5a429eab294d56af4288ff (127.21.214.65:41639), 8d7c03ac704b4cfab6c12b3274db129b (127.21.214.66:35121)
I20260811 01:44:05.920588 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:05.913496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:42025 (local address 127.21.214.65:41639)
0811 01:44:05.913760 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:05.913768 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:05.913790 (+ 22us) server_negotiation.cc:407] Connection header received
0811 01:44:05.913877 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:05.913882 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:05.913971 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:05.914084 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:05.914706 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.915627 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.917979 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.919491 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.919714 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:05.919792 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:05.920190 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 01:44:05.920323 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 01:44:05.920804 23540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:05.913076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:41639 (local address 127.21.214.67:42025)
0811 01:44:05.913507 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:05.913537 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:05.913682 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:05.914124 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:05.914129 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:05.914199 (+ 70us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:05.914462 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:05.914502 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.915793 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.915799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:05.917780 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:05.917793 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.919708 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.919713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:05.919994 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:05.920023 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 01:44:05.920129 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 01:44:05.920312 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260}
I20260811 01:44:05.922667 23171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656b5c06958243e7bdf5c2718dfc19e1" candidate_uuid: "af7cbe6d37d14c34bd9a9a4f8f02b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82119c8dfc5a429eab294d56af4288ff" is_pre_election: true
I20260811 01:44:05.923460 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:05.913091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:35121 (local address 127.21.214.67:59801)
0811 01:44:05.914413 (+ 1322us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:05.914430 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:05.914591 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:05.914999 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:05.915004 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:05.915032 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:05.915768 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:05.915780 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.917459 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.917463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:05.920039 (+ 2576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:05.920052 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.922588 (+ 2536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.922593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:05.922821 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:05.922891 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 01:44:05.922973 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 01:44:05.923069 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1148,"mutex_wait_us":165,"thread_start_us":686,"threads_started":1}
I20260811 01:44:05.923791 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:05.913625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:59801 (local address 127.21.214.66:35121)
0811 01:44:05.914002 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:05.914011 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:05.914571 (+ 560us) server_negotiation.cc:407] Connection header received
0811 01:44:05.914789 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:05.914794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:05.914857 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:05.914964 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:05.916063 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.917258 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.920262 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:05.922354 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:05.922548 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:05.922793 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:05.923051 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 01:44:05.923256 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198}
I20260811 01:44:05.923888 23171 raft_consensus.cc:2454] T 656b5c06958243e7bdf5c2718dfc19e1 P 82119c8dfc5a429eab294d56af4288ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af7cbe6d37d14c34bd9a9a4f8f02b94e in term 0.
I20260811 01:44:05.925112 23410 leader_election.cc:302] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82119c8dfc5a429eab294d56af4288ff, af7cbe6d37d14c34bd9a9a4f8f02b94e; no voters:
I20260811 01:44:05.925354 23330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656b5c06958243e7bdf5c2718dfc19e1" candidate_uuid: "af7cbe6d37d14c34bd9a9a4f8f02b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d7c03ac704b4cfab6c12b3274db129b" is_pre_election: true
I20260811 01:44:05.925799 23570 raft_consensus.cc:2781] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:44:05.926156 23570 raft_consensus.cc:482] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:44:05.926131 23330 raft_consensus.cc:2454] T 656b5c06958243e7bdf5c2718dfc19e1 P 8d7c03ac704b4cfab6c12b3274db129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af7cbe6d37d14c34bd9a9a4f8f02b94e in term 0.
I20260811 01:44:05.926353 23570 raft_consensus.cc:3037] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:05.929960 23570 raft_consensus.cc:504] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82119c8dfc5a429eab294d56af4288ff" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 41639 } } peers { permanent_uuid: "af7cbe6d37d14c34bd9a9a4f8f02b94e" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 44449 } } peers { permanent_uuid: "8d7c03ac704b4cfab6c12b3274db129b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 35121 } }
I20260811 01:44:05.930680 23570 leader_election.cc:288] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e [CANDIDATE]: Term 1 election: Requested vote from peers 82119c8dfc5a429eab294d56af4288ff (127.21.214.65:41639), 8d7c03ac704b4cfab6c12b3274db129b (127.21.214.66:35121)
I20260811 01:44:05.931249 23171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656b5c06958243e7bdf5c2718dfc19e1" candidate_uuid: "af7cbe6d37d14c34bd9a9a4f8f02b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82119c8dfc5a429eab294d56af4288ff"
I20260811 01:44:05.931264 23330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656b5c06958243e7bdf5c2718dfc19e1" candidate_uuid: "af7cbe6d37d14c34bd9a9a4f8f02b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d7c03ac704b4cfab6c12b3274db129b"
I20260811 01:44:05.931512 23171 raft_consensus.cc:3037] T 656b5c06958243e7bdf5c2718dfc19e1 P 82119c8dfc5a429eab294d56af4288ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:05.931581 23330 raft_consensus.cc:3037] T 656b5c06958243e7bdf5c2718dfc19e1 P 8d7c03ac704b4cfab6c12b3274db129b [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:05.935292 23171 raft_consensus.cc:2454] T 656b5c06958243e7bdf5c2718dfc19e1 P 82119c8dfc5a429eab294d56af4288ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af7cbe6d37d14c34bd9a9a4f8f02b94e in term 1.
I20260811 01:44:05.935489 23330 raft_consensus.cc:2454] T 656b5c06958243e7bdf5c2718dfc19e1 P 8d7c03ac704b4cfab6c12b3274db129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af7cbe6d37d14c34bd9a9a4f8f02b94e in term 1.
I20260811 01:44:05.935957 23410 leader_election.cc:302] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82119c8dfc5a429eab294d56af4288ff, af7cbe6d37d14c34bd9a9a4f8f02b94e; no voters:
I20260811 01:44:05.936337 23570 raft_consensus.cc:2781] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:05.937031 23570 raft_consensus.cc:686] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e [term 1 LEADER]: Becoming Leader. State: Replica: af7cbe6d37d14c34bd9a9a4f8f02b94e, State: Running, Role: LEADER
I20260811 01:44:05.937775 23570 consensus_queue.cc:237] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82119c8dfc5a429eab294d56af4288ff" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 41639 } } peers { permanent_uuid: "af7cbe6d37d14c34bd9a9a4f8f02b94e" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 44449 } } peers { permanent_uuid: "8d7c03ac704b4cfab6c12b3274db129b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 35121 } }
I20260811 01:44:05.944341 23007 catalog_manager.cc:5495] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e reported cstate change: term changed from 0 to 1, leader changed from <none> to af7cbe6d37d14c34bd9a9a4f8f02b94e (127.21.214.67). New cstate: current_term: 1 leader_uuid: "af7cbe6d37d14c34bd9a9a4f8f02b94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82119c8dfc5a429eab294d56af4288ff" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 41639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af7cbe6d37d14c34bd9a9a4f8f02b94e" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 44449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d7c03ac704b4cfab6c12b3274db129b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 35121 } health_report { overall_health: UNKNOWN } } }
W20260811 01:44:05.949821 23224 tablet.cc:2369] T 656b5c06958243e7bdf5c2718dfc19e1 P 82119c8dfc5a429eab294d56af4288ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:44:06.019945 23380 tablet.cc:2369] T 656b5c06958243e7bdf5c2718dfc19e1 P 8d7c03ac704b4cfab6c12b3274db129b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 01:44:06 dist-test-slave-9gcw krb5kdc[22959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:44:06.031033 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:06.007203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55894 (local address 127.21.214.65:41639)
0811 01:44:06.007568 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:06.007576 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:06.008280 (+ 704us) server_negotiation.cc:407] Connection header received
0811 01:44:06.008574 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:06.008586 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:06.008718 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:06.008856 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:06.011946 (+ 3090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:06.013225 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:06.014976 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:06.015898 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:06.016080 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:06.022820 (+ 6740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:06.022837 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:06.022854 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:06.022894 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:06.026667 (+ 3773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:06.027778 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:06.027787 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:06.027790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:06.027948 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:06.028505 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:06.028513 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:06.028516 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:06.029066 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:06.029581 (+ 515us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:06.030120 (+ 539us) server_negotiation.cc:299] Negotiation successful
0811 01:44:06.030474 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
Aug 11 01:44:06 dist-test-slave-9gcw krb5kdc[22959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412646, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 01:44:06 dist-test-slave-9gcw krb5kdc[22959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412646, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:44:06.092826 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:06.075022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55908 (local address 127.21.214.65:41639)
0811 01:44:06.075321 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:06.075338 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:06.075758 (+ 420us) server_negotiation.cc:407] Connection header received
0811 01:44:06.075912 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:06.075916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:06.075983 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:06.076073 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:06.078288 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:06.079029 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:06.080589 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:06.080986 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:06.081178 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:06.086220 (+ 5042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:06.086225 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:06.086229 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:06.086271 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:06.089709 (+ 3438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:06.090532 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:06.090536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:06.090539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:06.090732 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:06.091222 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:06.091226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:06.091228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:06.091556 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:06.091809 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:06.092167 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 01:44:06.092475 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 01:44:06 dist-test-slave-9gcw krb5kdc[22959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 01:44:06 dist-test-slave-9gcw krb5kdc[22959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:44:06.159528 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:06.140648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55918 (local address 127.21.214.65:41639)
0811 01:44:06.141097 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:06.141106 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:06.141534 (+ 428us) server_negotiation.cc:407] Connection header received
0811 01:44:06.141782 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:06.141808 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:06.141881 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:06.142005 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:06.144434 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:06.145300 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:06.146983 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:06.147481 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:06.147643 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:06.153482 (+ 5839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:06.153489 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:06.153492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:06.153535 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:06.156669 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:06.157488 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:06.157492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:06.157495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:06.157634 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:06.158018 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:06.158022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:06.158024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:06.158362 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:06.158565 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:06.158908 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 01:44:06.159118 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
W20260811 01:44:06.161795 23127 tablet_service.cc:2908] Rejecting scan request for tablet 656b5c06958243e7bdf5c2718dfc19e1: Uninitialized: clean time has not yet been initialized
I20260811 01:44:06.165117 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 23075
I20260811 01:44:06.181365 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:06.166142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55926 (local address 127.21.214.65:41639)
0811 01:44:06.166339 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:06.166344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:06.166800 (+ 456us) server_negotiation.cc:407] Connection header received
0811 01:44:06.167017 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:06.167021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:06.167107 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:06.167207 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:06.168735 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:06.169614 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:06.171152 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:06.171611 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:06.171802 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:06.174433 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:06.174438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:06.174442 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:06.174476 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:06.178387 (+ 3911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:06.179078 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:06.179082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:06.179085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:06.179320 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:06.179693 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:06.179697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:06.179699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:06.180171 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:06.180491 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:06.180684 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 01:44:06.180897 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 01:44:06.182126 23218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:06.287072 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 23075
I20260811 01:44:06.311028 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 23229
Aug 11 01:44:06 dist-test-slave-9gcw krb5kdc[22959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.66@KRBTEST.COM
I20260811 01:44:06.330101 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:06.312085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34868 (local address 127.21.214.66:35121)
0811 01:44:06.312297 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:06.312302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:06.312323 (+ 21us) server_negotiation.cc:407] Connection header received
0811 01:44:06.312408 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:06.312413 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:06.312482 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:06.312570 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:06.314302 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:06.315214 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:06.316823 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:06.317439 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:06.317547 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:06.322325 (+ 4778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:06.322344 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:06.322362 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:06.322410 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:06.326641 (+ 4231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:06.327748 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:06.327759 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:06.327762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:06.327900 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:06.328410 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:06.328417 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:06.328424 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:06.328872 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:06.329114 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:06.329481 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 01:44:06.329712 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 01:44:06.330829 23374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:06.419410 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:06.418820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:41639 (local address 127.21.214.67:34895)
0811 01:44:06.419034 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:06.419249 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 01:44:06.419745 23410 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.214.65:41639: connect: Connection refused (error 111)
I20260811 01:44:06.421612 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:06.421102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:41639 (local address 127.21.214.67:37973)
0811 01:44:06.421337 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:06.421439 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260811 01:44:06.421849 23410 consensus_peers.cc:487] T 656b5c06958243e7bdf5c2718dfc19e1 P af7cbe6d37d14c34bd9a9a4f8f02b94e -> Peer 82119c8dfc5a429eab294d56af4288ff (127.21.214.65:41639): Couldn't send request to peer 82119c8dfc5a429eab294d56af4288ff. Status: Network error: Client connection negotiation failed: client connection to 127.21.214.65:41639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 01:44:06.423928 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 23229
I20260811 01:44:06.447180 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 23386
Aug 11 01:44:06 dist-test-slave-9gcw krb5kdc[22959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.67@KRBTEST.COM
I20260811 01:44:06.468442 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:06.448101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55980 (local address 127.21.214.67:44449)
0811 01:44:06.448667 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:06.448671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:06.448692 (+ 21us) server_negotiation.cc:407] Connection header received
0811 01:44:06.448761 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:06.448766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:06.448835 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:06.448929 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:06.450309 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:06.451010 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:06.452326 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:06.452718 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:06.452814 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:06.462007 (+ 9193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:06.462030 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:06.462052 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:06.462089 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:06.464949 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:06.466172 (+ 1223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:06.466182 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:06.466184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:06.466346 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:06.466764 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:06.466776 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:06.466779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:06.467181 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:06.467517 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:06.467829 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 01:44:06.468083 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":286,"threads_started":1}
I20260811 01:44:06.469230 23531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:06.562738 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:06.562263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:35121 (local address 127.21.214.67:56799)
0811 01:44:06.562489 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:06.562591 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.66:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 01:44:06.563199 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 23386
I20260811 01:44:06.582963 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 22973
I20260811 01:44:06.584338 23040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:06.697285 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 22973
2026-08-11T01:44:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4819 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:44:06 dist-test-slave-9gcw krb5kdc[23600](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:44:06 dist-test-slave-9gcw krb5kdc[23600](info): set up 2 sockets
Aug 11 01:44:06 dist-test-slave-9gcw krb5kdc[23600](info): commencing operation
krb5kdc: starting...
W20260811 01:44:07.941368 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:44:08 dist-test-slave-9gcw krb5kdc[23600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:44:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:44:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:44:08.170379 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:36965
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:36965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:08.515553 23614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:08.515944 23614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:08.516110 23614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:08.523384 23614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:44:08.523537 23614 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 01:44:08.523633 23614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:08.523761 23614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:08.523851 23614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:44:08.523943 23614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:44:08.534013 23614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:36965
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:36965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:08.535449 23614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:08.536839 23614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:08.549381 23623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:08.549463 23620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:08.549410 23621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:08.550474 23614 server_base.cc:1034] running on GCE node
I20260811 01:44:08.551330 23614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:08.552830 23614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:08.554096 23614 hybrid_clock.cc:648] HybridClock initialized: now 1786412648554002 us; error 66 us; skew 500 ppm
I20260811 01:44:08.554466 23614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:08 dist-test-slave-9gcw krb5kdc[23600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412648, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:08.557727 23614 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:44:08.559331 23614 webserver.cc:460] Webserver started at http://127.21.214.126:34529/ using document root <none> and password file <none>
I20260811 01:44:08.560066 23614 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:08.560212 23614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:08.560551 23614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:08.563714 23614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "b5598d5d440d4861b80cbb6b3a5721f3"
format_stamp: "Formatted at 2026-08-11 01:44:08 on dist-test-slave-9gcw"
server_key: "9fca580c7e91a723fa7a520d6300168b"
server_key_iv: "bae441b696ae9bbb603626b5afde380e"
server_key_version: "encryptionkey@0"
I20260811 01:44:08.564534 23614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "b5598d5d440d4861b80cbb6b3a5721f3"
format_stamp: "Formatted at 2026-08-11 01:44:08 on dist-test-slave-9gcw"
server_key: "9fca580c7e91a723fa7a520d6300168b"
server_key_iv: "bae441b696ae9bbb603626b5afde380e"
server_key_version: "encryptionkey@0"
I20260811 01:44:08.570032 23614 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 01:44:08.573967 23631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:08.575408 23614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:44:08.575598 23614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "b5598d5d440d4861b80cbb6b3a5721f3"
format_stamp: "Formatted at 2026-08-11 01:44:08 on dist-test-slave-9gcw"
server_key: "9fca580c7e91a723fa7a520d6300168b"
server_key_iv: "bae441b696ae9bbb603626b5afde380e"
server_key_version: "encryptionkey@0"
I20260811 01:44:08.575839 23614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:08.593106 23614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:08.597605 23614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:08.597904 23614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:08.621507 23614 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:36965
I20260811 01:44:08.621542 23697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:36965 every 8 connection(s)
I20260811 01:44:08.623189 23614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:44:08.627902 23698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:08.632550 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 23614
I20260811 01:44:08.632807 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:44:08.633575 22361 external_mini_cluster.cc:1428] Setting key b5e0722654bb8d09d0507827492a3ca1
I20260811 01:44:08.641537 23698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3: Bootstrap starting.
I20260811 01:44:08.645772 23698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:08.647094 23698 log.cc:824] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:08.650545 23698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3: No bootstrap required, opened a new log
Aug 11 01:44:08 dist-test-slave-9gcw krb5kdc[23600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:08.662002 23701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:08.635945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33734 (local address 127.21.214.126:36965)
0811 01:44:08.636886 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:08.636908 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:08.636965 (+ 57us) server_negotiation.cc:407] Connection header received
0811 01:44:08.639667 (+ 2702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:08.639704 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:08.640316 (+ 612us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:08.640826 (+ 510us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:08.642629 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:08.644815 (+ 2186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:08.646410 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:08.647584 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:08.647845 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:08.652378 (+ 4533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:08.652397 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:08.652425 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:08.652479 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:08.656881 (+ 4402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:08.657751 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:08.657761 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:08.657773 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:08.658014 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:08.658391 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:08.658396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:08.658399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:08.659250 (+ 851us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:08.659593 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:08.659927 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 01:44:08.660441 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":438,"threads_started":1}
I20260811 01:44:08.662166 23698 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5598d5d440d4861b80cbb6b3a5721f3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36965 } }
I20260811 01:44:08.662645 23698 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:08.662788 23698 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5598d5d440d4861b80cbb6b3a5721f3, State: Initialized, Role: FOLLOWER
I20260811 01:44:08.663398 23698 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5598d5d440d4861b80cbb6b3a5721f3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36965 } }
I20260811 01:44:08.663698 23698 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:44:08.663862 23698 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:44:08.664068 23698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:08.666553 23698 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5598d5d440d4861b80cbb6b3a5721f3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36965 } }
I20260811 01:44:08.667115 23698 leader_election.cc:302] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5598d5d440d4861b80cbb6b3a5721f3; no voters:
I20260811 01:44:08.667752 23698 leader_election.cc:288] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:44:08.667878 23703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:08.669317 23703 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3 [term 1 LEADER]: Becoming Leader. State: Replica: b5598d5d440d4861b80cbb6b3a5721f3, State: Running, Role: LEADER
I20260811 01:44:08.669869 23703 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5598d5d440d4861b80cbb6b3a5721f3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36965 } }
I20260811 01:44:08.670169 23698 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:44:08.673317 23704 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5598d5d440d4861b80cbb6b3a5721f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5598d5d440d4861b80cbb6b3a5721f3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36965 } } }
I20260811 01:44:08.673910 23704 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3 [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:08.674434 23707 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5598d5d440d4861b80cbb6b3a5721f3. Latest consensus state: current_term: 1 leader_uuid: "b5598d5d440d4861b80cbb6b3a5721f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5598d5d440d4861b80cbb6b3a5721f3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36965 } } }
I20260811 01:44:08.674836 23707 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3 [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:08.675168 23717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:44:08.684542 23717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:44:08.694574 23717 catalog_manager.cc:1347] Generated new cluster ID: c2cb78ae86704f97a772746a32037b7b
I20260811 01:44:08.694716 23717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:44:08.723537 23717 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:44:08.725724 23717 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:44:08.734184 23717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3: Generated new TSK 0
I20260811 01:44:08.735416 23717 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
I20260811 01:44:08.793855 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:36965
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:09.146868 23734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:09.147171 23734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:09.147274 23734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:09.155121 23734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:09.155252 23734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:09.155381 23734 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 01:44:09.155472 23734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:44:09.166057 23734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.214.126:36965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:09.167430 23734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:09.168824 23734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:09.183279 23743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:09.183284 23741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:09.183917 23740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:09.184154 23734 server_base.cc:1034] running on GCE node
I20260811 01:44:09.184860 23734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:09.186071 23734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:09.187335 23734 hybrid_clock.cc:648] HybridClock initialized: now 1786412649187255 us; error 49 us; skew 500 ppm
I20260811 01:44:09.187693 23734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:09 dist-test-slave-9gcw krb5kdc[23600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412649, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:09.191393 23734 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:44:09.193133 23734 webserver.cc:460] Webserver started at http://127.21.214.65:44185/ using document root <none> and password file <none>
I20260811 01:44:09.194000 23734 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:09.194097 23734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:09.194450 23734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:09.197736 23734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "0901bf3969a04adb8f28852b35c0c350"
format_stamp: "Formatted at 2026-08-11 01:44:09 on dist-test-slave-9gcw"
server_key: "5fdd9dda398d31cdb6091b714754657b"
server_key_iv: "044737b7a20a2e74d3ce6230667ed6c1"
server_key_version: "encryptionkey@0"
I20260811 01:44:09.198599 23734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "0901bf3969a04adb8f28852b35c0c350"
format_stamp: "Formatted at 2026-08-11 01:44:09 on dist-test-slave-9gcw"
server_key: "5fdd9dda398d31cdb6091b714754657b"
server_key_iv: "044737b7a20a2e74d3ce6230667ed6c1"
server_key_version: "encryptionkey@0"
I20260811 01:44:09.204213 23734 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 01:44:09.208019 23750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:09.209517 23734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 01:44:09.209714 23734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "0901bf3969a04adb8f28852b35c0c350"
format_stamp: "Formatted at 2026-08-11 01:44:09 on dist-test-slave-9gcw"
server_key: "5fdd9dda398d31cdb6091b714754657b"
server_key_iv: "044737b7a20a2e74d3ce6230667ed6c1"
server_key_version: "encryptionkey@0"
I20260811 01:44:09.209986 23734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:09.226727 23734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:09.231164 23734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:09.231458 23734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:09.234800 23734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:09.234917 23734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:09.235118 23734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:09.235268 23734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 11 01:44:09 dist-test-slave-9gcw krb5kdc[23600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412649, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:09.262668 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:09.235301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:58105 (local address 127.21.214.126:36965)
0811 01:44:09.235894 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:09.235902 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:09.237414 (+ 1512us) server_negotiation.cc:407] Connection header received
0811 01:44:09.239753 (+ 2339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:09.239761 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:09.239873 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:09.240011 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:09.242669 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:09.244004 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:09.246987 (+ 2983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:09.247464 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:09.247615 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:09.253844 (+ 6229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:09.253850 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:09.253853 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:09.253887 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:09.257655 (+ 3768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:09.258815 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:09.258824 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:09.258829 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:09.259001 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:09.259736 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:09.259743 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:09.259752 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:09.260176 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:09.260337 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:09.261719 (+ 1382us) server_negotiation.cc:299] Negotiation successful
0811 01:44:09.262109 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":273,"threads_started":1}
I20260811 01:44:09.263384 23760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:09.236165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36965 (local address 127.21.214.65:58105)
0811 01:44:09.237193 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:09.237250 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:09.239421 (+ 2171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:09.240295 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:09.240305 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:09.241239 (+ 934us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:09.242406 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:09.242431 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:09.244226 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:09.244237 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:09.246800 (+ 2563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:09.246813 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:09.247750 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:09.247758 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:09.247987 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:09.248863 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:09.248902 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:09.253532 (+ 4630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:09.257815 (+ 4283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:09.257825 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:09.257848 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:09.258593 (+ 745us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:09.259112 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:09.259124 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:09.259128 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:09.259548 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:09.260406 (+ 858us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:09.260413 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:09.261064 (+ 651us) client_negotiation.cc:769] Sending connection context
0811 01:44:09.261544 (+ 480us) client_negotiation.cc:240] Negotiation successful
0811 01:44:09.262094 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":427,"threads_started":1}
I20260811 01:44:09.290889 23840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:09.272444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36965 (local address 127.21.214.65:47961)
0811 01:44:09.273229 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:09.273256 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:09.273459 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:09.273951 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:09.273959 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:09.274774 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:09.275363 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:09.275383 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:09.276616 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:09.276624 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:09.278976 (+ 2352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:09.278999 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:09.279848 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:09.279883 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:09.280124 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:09.281005 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:09.281038 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:09.282678 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:09.287198 (+ 4520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:09.287203 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:09.287209 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:09.288005 (+ 796us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:09.288734 (+ 729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:09.288741 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:09.288746 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:09.289018 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:09.289917 (+ 899us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:09.289921 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:09.290104 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 01:44:09.290240 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 01:44:09.290529 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":372,"threads_started":1}
I20260811 01:44:09.291286 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:09.272504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:47961 (local address 127.21.214.126:36965)
0811 01:44:09.272768 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:09.272777 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:09.273387 (+ 610us) server_negotiation.cc:407] Connection header received
0811 01:44:09.273625 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:09.273630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:09.273777 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:09.273871 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:09.275540 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:09.276401 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:09.279157 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:09.279588 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:09.279759 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:09.282955 (+ 3196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:09.282967 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:09.282975 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:09.283029 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:09.286968 (+ 3939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:09.288270 (+ 1302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:09.288292 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:09.288298 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:09.288547 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:09.289231 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:09.289238 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:09.289244 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:09.289742 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:09.289957 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:09.290423 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 01:44:09.290753 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 01:44:09.298735 23734 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:34025
I20260811 01:44:09.298780 23884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:34025 every 8 connection(s)
I20260811 01:44:09.299971 23734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:44:09.302276 23885 heartbeater.cc:347] Connected to a master server at 127.21.214.126:36965
I20260811 01:44:09.302525 23885 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:09.303315 23885 heartbeater.cc:511] Master 127.21.214.126:36965 requested a full tablet report, sending...
I20260811 01:44:09.305847 23648 ts_manager.cc:194] Registered new tserver with Master: 0901bf3969a04adb8f28852b35c0c350 (127.21.214.65:34025)
I20260811 01:44:09.307090 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 23734
I20260811 01:44:09.307374 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:44:09.308367 22361 external_mini_cluster.cc:1428] Setting key 75f7b7f013a71be79c23315b6d7e4f51
I20260811 01:44:09.308404 23648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:58105
WARNING: no policy specified for kudu/127.21.214.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
I20260811 01:44:09.357790 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:36965
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:09.699553 23889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:09.699950 23889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:09.700122 23889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:09.707823 23889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:09.707986 23889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:09.708138 23889 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 01:44:09.708282 23889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:44:09.718918 23889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.214.126:36965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:09.720381 23889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:09.722023 23889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:09.735934 23895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:09.736003 23898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:09.736413 23902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:09.736806 23889 server_base.cc:1034] running on GCE node
I20260811 01:44:09.737545 23889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:09.738533 23889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:09.739801 23889 hybrid_clock.cc:648] HybridClock initialized: now 1786412649739738 us; error 38 us; skew 500 ppm
I20260811 01:44:09.740130 23889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:09 dist-test-slave-9gcw krb5kdc[23600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412649, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:09.743696 23889 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:44:09.745328 23889 webserver.cc:460] Webserver started at http://127.21.214.66:44227/ using document root <none> and password file <none>
I20260811 01:44:09.746088 23889 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:09.746214 23889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:09.746567 23889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:09.749715 23889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "01aa5bcc010e4b228017c9f18af9cfa8"
format_stamp: "Formatted at 2026-08-11 01:44:09 on dist-test-slave-9gcw"
server_key: "cd52e8724130462ba39186a0e9a7ef72"
server_key_iv: "29180221764acab7d0d5b0d086304607"
server_key_version: "encryptionkey@0"
I20260811 01:44:09.750618 23889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "01aa5bcc010e4b228017c9f18af9cfa8"
format_stamp: "Formatted at 2026-08-11 01:44:09 on dist-test-slave-9gcw"
server_key: "cd52e8724130462ba39186a0e9a7ef72"
server_key_iv: "29180221764acab7d0d5b0d086304607"
server_key_version: "encryptionkey@0"
I20260811 01:44:09.756039 23889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 01:44:09.759825 23910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:09.761374 23889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:44:09.761564 23889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "01aa5bcc010e4b228017c9f18af9cfa8"
format_stamp: "Formatted at 2026-08-11 01:44:09 on dist-test-slave-9gcw"
server_key: "cd52e8724130462ba39186a0e9a7ef72"
server_key_iv: "29180221764acab7d0d5b0d086304607"
server_key_version: "encryptionkey@0"
I20260811 01:44:09.761792 23889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:09.791365 23889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:09.796387 23889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:09.796744 23889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:09.799629 23889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:09.799754 23889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:09.799942 23889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:09.800025 23889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 01:44:09 dist-test-slave-9gcw krb5kdc[23600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412649, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:09.822996 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:09.800360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:46753 (local address 127.21.214.126:36965)
0811 01:44:09.801098 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:09.801104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:09.802434 (+ 1330us) server_negotiation.cc:407] Connection header received
0811 01:44:09.804601 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:09.804607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:09.804716 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:09.804855 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:09.807339 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:09.808178 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:09.810609 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:09.811244 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:09.811410 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:09.816219 (+ 4809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:09.816225 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:09.816228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:09.816268 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:09.819453 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:09.820221 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:09.820228 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:09.820236 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:09.820372 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:09.820839 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:09.820843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:09.820845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:09.821125 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:09.821355 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:09.822359 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0811 01:44:09.822583 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":493,"threads_started":1}
I20260811 01:44:09.824558 23920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:09.801226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36965 (local address 127.21.214.66:46753)
0811 01:44:09.802228 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:09.802283 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:09.804231 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:09.805099 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:09.805108 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:09.806102 (+ 994us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:09.807133 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:09.807156 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:09.808352 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:09.808358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:09.810341 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:09.810364 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:09.811740 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:09.811745 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:09.812048 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:09.813032 (+ 984us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:09.813075 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:09.815975 (+ 2900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:09.819594 (+ 3619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:09.819604 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:09.819621 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:09.820084 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:09.820489 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:09.820494 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:09.820497 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:09.820705 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:09.821328 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:09.821335 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:09.821919 (+ 584us) client_negotiation.cc:769] Sending connection context
0811 01:44:09.822245 (+ 326us) client_negotiation.cc:240] Negotiation successful
0811 01:44:09.822855 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":424,"threads_started":1}
I20260811 01:44:09.850991 23889 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:43163
I20260811 01:44:09.851086 24048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:43163 every 8 connection(s)
I20260811 01:44:09.851783 23992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:09.833029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36965 (local address 127.21.214.66:56627)
0811 01:44:09.833893 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:09.833919 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:09.834167 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:09.834817 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:09.834824 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:09.835787 (+ 963us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:09.836402 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:09.836426 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:09.838182 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:09.838191 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:09.840449 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:09.840467 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:09.841321 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:09.841331 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:09.841573 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:09.842471 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:09.842504 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:09.844088 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:09.848310 (+ 4222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:09.848317 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:09.848327 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:09.848993 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:09.849459 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:09.849462 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:09.849464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:09.849751 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:09.850636 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:09.850639 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:09.850914 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 01:44:09.851091 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 01:44:09.851461 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":490,"threads_started":1}
I20260811 01:44:09.852641 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:09.832921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:56627 (local address 127.21.214.126:36965)
0811 01:44:09.833131 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:09.833173 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:09.834024 (+ 851us) server_negotiation.cc:407] Connection header received
0811 01:44:09.834378 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:09.834384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:09.834486 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:09.834571 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:09.836633 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:09.837993 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:09.840607 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:09.841041 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:09.841218 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:09.844344 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:09.844350 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:09.844353 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:09.844388 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:09.848090 (+ 3702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:09.849124 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:09.849130 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:09.849134 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:09.849339 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:09.849907 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:09.849915 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:09.849922 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:09.850482 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:09.850650 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:09.852099 (+ 1449us) server_negotiation.cc:299] Negotiation successful
0811 01:44:09.852296 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 01:44:09.852800 23889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:44:09.855182 24049 heartbeater.cc:347] Connected to a master server at 127.21.214.126:36965
I20260811 01:44:09.855495 24049 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:09.856328 24049 heartbeater.cc:511] Master 127.21.214.126:36965 requested a full tablet report, sending...
I20260811 01:44:09.857752 23648 ts_manager.cc:194] Registered new tserver with Master: 01aa5bcc010e4b228017c9f18af9cfa8 (127.21.214.66:43163)
I20260811 01:44:09.858562 23648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:46753
I20260811 01:44:09.860227 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 23889
I20260811 01:44:09.860453 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:44:09.860985 22361 external_mini_cluster.cc:1428] Setting key e778c2586b1a6c0189bbac8ac38dc558
WARNING: no policy specified for kudu/127.21.214.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
I20260811 01:44:09.909974 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:36965
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:10.256057 24054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:10.256366 24054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:10.256553 24054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:10.263864 24054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:10.263989 24054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:10.264174 24054 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 01:44:10.264303 24054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:44:10.274796 24054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.214.126:36965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:10.276438 24054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:10.277959 24054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:10.292379 24060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:10.292554 24059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:10.292294 24062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:10.292865 24054 server_base.cc:1034] running on GCE node
I20260811 01:44:10.293646 24054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:10.294641 24054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:10.295881 24054 hybrid_clock.cc:648] HybridClock initialized: now 1786412650295810 us; error 37 us; skew 500 ppm
I20260811 01:44:10.296209 24054 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:10 dist-test-slave-9gcw krb5kdc[23600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412650, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:10.299801 24054 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:44:10.301533 24054 webserver.cc:460] Webserver started at http://127.21.214.67:41233/ using document root <none> and password file <none>
I20260811 01:44:10.302325 24054 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:10.302416 24054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:10.302791 24054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:10.305769 24054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "4b2781914c654d33a3d8e9228a1c7d1c"
format_stamp: "Formatted at 2026-08-11 01:44:10 on dist-test-slave-9gcw"
server_key: "8bbbd8ed617759f4f02e5ba831e6542d"
server_key_iv: "370d214d74d40390d463da0bb4e53605"
server_key_version: "encryptionkey@0"
I20260811 01:44:10.306588 24054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "4b2781914c654d33a3d8e9228a1c7d1c"
format_stamp: "Formatted at 2026-08-11 01:44:10 on dist-test-slave-9gcw"
server_key: "8bbbd8ed617759f4f02e5ba831e6542d"
server_key_iv: "370d214d74d40390d463da0bb4e53605"
server_key_version: "encryptionkey@0"
I20260811 01:44:10.311887 23885 heartbeater.cc:503] Master 127.21.214.126:36965 was elected leader, sending a full tablet report...
I20260811 01:44:10.312116 24054 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 01:44:10.316007 24070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:10.317451 24054 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:44:10.317625 24054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "4b2781914c654d33a3d8e9228a1c7d1c"
format_stamp: "Formatted at 2026-08-11 01:44:10 on dist-test-slave-9gcw"
server_key: "8bbbd8ed617759f4f02e5ba831e6542d"
server_key_iv: "370d214d74d40390d463da0bb4e53605"
server_key_version: "encryptionkey@0"
I20260811 01:44:10.317838 24054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:10.342273 24054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:10.347429 24054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:10.347723 24054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:10.350836 24054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:10.350955 24054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:10.351130 24054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:10.351274 24054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 11 01:44:10 dist-test-slave-9gcw krb5kdc[23600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412650, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:10.378640 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:10.351351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:39241 (local address 127.21.214.126:36965)
0811 01:44:10.351581 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:10.351592 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:10.353208 (+ 1616us) server_negotiation.cc:407] Connection header received
0811 01:44:10.355422 (+ 2214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:10.355428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:10.355530 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:10.355679 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:10.358336 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.359415 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.362150 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.362856 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.363025 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:10.368789 (+ 5764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:10.368799 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:10.368806 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:10.368858 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:10.373219 (+ 4361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:10.374575 (+ 1356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:10.374581 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:10.374585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:10.374776 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:10.375545 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:10.375551 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:10.375556 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:10.376055 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:10.376219 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:10.377808 (+ 1589us) server_negotiation.cc:299] Negotiation successful
0811 01:44:10.378137 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 01:44:10.382519 24082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:10.352054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36965 (local address 127.21.214.67:39241)
0811 01:44:10.352970 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:10.353033 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:10.355082 (+ 2049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:10.356069 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:10.356080 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:10.357113 (+ 1033us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:10.358131 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:10.358152 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.359620 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.359631 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:10.361932 (+ 2301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:10.361953 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.363174 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.363184 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:10.363419 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:10.364285 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:10.364347 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:10.368431 (+ 4084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:10.373408 (+ 4977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:10.373426 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:10.373455 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:10.374360 (+ 905us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:10.374991 (+ 631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:10.374998 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:10.375003 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:10.375367 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:10.376495 (+ 1128us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:10.376510 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:10.377223 (+ 713us) client_negotiation.cc:769] Sending connection context
0811 01:44:10.379180 (+ 1957us) client_negotiation.cc:240] Negotiation successful
0811 01:44:10.380157 (+ 977us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":408,"threads_started":1}
I20260811 01:44:10.395859 24054 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:33957
I20260811 01:44:10.395888 24214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:33957 every 8 connection(s)
I20260811 01:44:10.397634 24054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:44:10.400034 24215 heartbeater.cc:347] Connected to a master server at 127.21.214.126:36965
I20260811 01:44:10.400511 24215 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:10.401367 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 24054
I20260811 01:44:10.401445 24215 heartbeater.cc:511] Master 127.21.214.126:36965 requested a full tablet report, sending...
I20260811 01:44:10.401579 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:44:10.402031 22361 external_mini_cluster.cc:1428] Setting key a191f2c74b5d73deda0471821bcc7e07
I20260811 01:44:10.403239 23648 ts_manager.cc:194] Registered new tserver with Master: 4b2781914c654d33a3d8e9228a1c7d1c (127.21.214.67:33957)
I20260811 01:44:10.404484 23648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:39241
I20260811 01:44:10.405684 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:44:10.407429 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:10.391394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:36643 (local address 127.21.214.126:36965)
0811 01:44:10.391629 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:10.391640 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:10.392319 (+ 679us) server_negotiation.cc:407] Connection header received
0811 01:44:10.392585 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:10.392593 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:10.392700 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:10.392809 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:10.394141 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.395263 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.397429 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.397934 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.398041 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:10.400222 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:10.400227 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:10.400230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:10.400263 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:10.404126 (+ 3863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:10.405061 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:10.405067 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:10.405071 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:10.405303 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:10.405773 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:10.405778 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:10.405781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:10.406137 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:10.406291 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:10.406872 (+ 581us) server_negotiation.cc:299] Negotiation successful
0811 01:44:10.407097 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 01:44:10.407589 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:10.391330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36965 (local address 127.21.214.67:36643)
0811 01:44:10.392159 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:10.392184 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:10.392380 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:10.392844 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:10.392849 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:10.393625 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:10.393968 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:10.393980 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.395482 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.395490 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:10.397265 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:10.397282 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.398154 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.398168 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:10.398430 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:10.399265 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:10.399284 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:10.400031 (+ 747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:10.404308 (+ 4277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:10.404313 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:10.404317 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:10.404889 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:10.405441 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:10.405445 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:10.405449 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:10.405626 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:10.406283 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:10.406288 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:10.406550 (+ 262us) client_negotiation.cc:769] Sending connection context
0811 01:44:10.406748 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 01:44:10.407099 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":24,"thread_start_us":406,"threads_started":1}
Aug 11 01:44:10 dist-test-slave-9gcw krb5kdc[23600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412650, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 01:44:10 dist-test-slave-9gcw krb5kdc[23600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412650, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:10.460547 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:10.445762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33750 (local address 127.21.214.126:36965)
0811 01:44:10.446027 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:10.446034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:10.446323 (+ 289us) server_negotiation.cc:407] Connection header received
0811 01:44:10.446567 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:10.446572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:10.446640 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:10.446761 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:10.448202 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.448973 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.450332 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.450764 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.450909 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:10.455068 (+ 4159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:10.455073 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:10.455076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:10.455114 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:10.457804 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:10.458510 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:10.458515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:10.458517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:10.458655 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:10.459067 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:10.459072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:10.459077 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:10.459373 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:10.459654 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:10.459966 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 01:44:10.460205 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 01:44:10.464915 23647 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:44:10.472502 23647 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:44:10.503639 24230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:10.487444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:33957 (local address 127.0.0.1:34366)
0811 01:44:10.490133 (+ 2689us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:10.490192 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:10.490405 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:10.491803 (+ 1398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:10.491816 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:10.491861 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:10.492400 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:10.492415 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.495321 (+ 2906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.495351 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:10.498903 (+ 3552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:10.498930 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.502296 (+ 3366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.502303 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:10.502631 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:10.502707 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 01:44:10.502882 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 01:44:10.503147 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2066,"thread_start_us":1895,"threads_started":1}
I20260811 01:44:10.504477 24232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:10.487823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34366 (local address 127.21.214.67:33957)
0811 01:44:10.488657 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:10.488667 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:10.490633 (+ 1966us) server_negotiation.cc:407] Connection header received
0811 01:44:10.490775 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:10.490788 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:10.491119 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:10.491344 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:10.492704 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.495072 (+ 2368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.499381 (+ 4309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.501586 (+ 2205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.501838 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:10.502162 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:10.503977 (+ 1815us) server_negotiation.cc:299] Negotiation successful
0811 01:44:10.504167 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":465,"threads_started":1}
I20260811 01:44:10.509002 24132 tablet_service.cc:1467] Processing CreateTablet for tablet 2d32364dfa0e484f96f2585c07f339b0 (DEFAULT_TABLE table=test-table [id=793f414ae6bc454b8341f02b0bc5b1cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:10.510828 24132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d32364dfa0e484f96f2585c07f339b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:10.513676 24234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:10.490041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:43163 (local address 127.0.0.1:40900)
0811 01:44:10.503214 (+ 13173us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:10.503232 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:10.503442 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:10.505037 (+ 1595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:10.505041 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:10.505067 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:10.505787 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:10.505803 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.507727 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.507733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:10.510428 (+ 2695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:10.510446 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.512719 (+ 2273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.512726 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:10.512934 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:10.512967 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 01:44:10.513067 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 01:44:10.513263 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13021,"mutex_wait_us":79,"thread_start_us":12785,"threads_started":1}
I20260811 01:44:10.513736 24236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:10.490497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40900 (local address 127.21.214.66:43163)
0811 01:44:10.504321 (+ 13824us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:10.504331 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:10.504362 (+ 31us) server_negotiation.cc:407] Connection header received
0811 01:44:10.504461 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:10.504469 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:10.504732 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:10.504968 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:10.506049 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.507514 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.510629 (+ 3115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.512184 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.512332 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:10.512554 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:10.513217 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 01:44:10.513412 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13101,"thread_start_us":13576,"threads_started":1}
I20260811 01:44:10.513916 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:10.490535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55920 (local address 127.21.214.65:34025)
0811 01:44:10.502577 (+ 12042us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:10.502590 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:10.504012 (+ 1422us) server_negotiation.cc:407] Connection header received
0811 01:44:10.504170 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:10.504180 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:10.504482 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:10.505314 (+ 832us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:10.506114 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.507471 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.510045 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.511570 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.511735 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:10.511961 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:10.513296 (+ 1335us) server_negotiation.cc:299] Negotiation successful
0811 01:44:10.513535 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11805,"thread_start_us":11592,"threads_started":1}
I20260811 01:44:10.514325 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:10.489716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:34025 (local address 127.0.0.1:55920)
0811 01:44:10.503012 (+ 13296us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:10.503039 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:10.503227 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:10.505321 (+ 2094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:10.505327 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:10.505364 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:10.505907 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:10.505923 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.507691 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.507698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:10.509852 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:10.509874 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.512137 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.512145 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:10.512354 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:10.512392 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 01:44:10.512490 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 01:44:10.512659 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13137,"mutex_wait_us":97,"thread_start_us":12890,"threads_started":1}
I20260811 01:44:10.517901 23962 tablet_service.cc:1467] Processing CreateTablet for tablet 2d32364dfa0e484f96f2585c07f339b0 (DEFAULT_TABLE table=test-table [id=793f414ae6bc454b8341f02b0bc5b1cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:10.518215 23805 tablet_service.cc:1467] Processing CreateTablet for tablet 2d32364dfa0e484f96f2585c07f339b0 (DEFAULT_TABLE table=test-table [id=793f414ae6bc454b8341f02b0bc5b1cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:10.519557 23962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d32364dfa0e484f96f2585c07f339b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:10.519788 23805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d32364dfa0e484f96f2585c07f339b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:10.528293 24237 tablet_bootstrap.cc:492] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c: Bootstrap starting.
I20260811 01:44:10.533008 24237 tablet_bootstrap.cc:654] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:10.534296 24238 tablet_bootstrap.cc:492] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8: Bootstrap starting.
I20260811 01:44:10.534942 24237 log.cc:824] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:10.539273 24237 tablet_bootstrap.cc:492] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c: No bootstrap required, opened a new log
I20260811 01:44:10.539710 24237 ts_tablet_manager.cc:1397] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260811 01:44:10.540158 24238 tablet_bootstrap.cc:654] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:10.542596 24238 log.cc:824] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:10.537375 24239 tablet_bootstrap.cc:492] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350: Bootstrap starting.
I20260811 01:44:10.548173 24239 tablet_bootstrap.cc:654] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:10.549942 24239 log.cc:824] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:10.551509 24237 raft_consensus.cc:348] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } }
I20260811 01:44:10.552103 24237 raft_consensus.cc:374] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:10.552304 24237 raft_consensus.cc:729] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b2781914c654d33a3d8e9228a1c7d1c, State: Initialized, Role: FOLLOWER
I20260811 01:44:10.553081 24238 tablet_bootstrap.cc:492] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8: No bootstrap required, opened a new log
I20260811 01:44:10.553521 24238 ts_tablet_manager.cc:1397] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.009s
I20260811 01:44:10.553438 24237 consensus_queue.cc:260] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } }
I20260811 01:44:10.555217 24215 heartbeater.cc:503] Master 127.21.214.126:36965 was elected leader, sending a full tablet report...
I20260811 01:44:10.555981 24239 tablet_bootstrap.cc:492] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350: No bootstrap required, opened a new log
I20260811 01:44:10.556313 24237 ts_tablet_manager.cc:1428] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c: Time spent starting tablet: real 0.016s user 0.014s sys 0.004s
I20260811 01:44:10.556501 24239 ts_tablet_manager.cc:1397] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.000s
I20260811 01:44:10.567688 24239 raft_consensus.cc:348] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } }
I20260811 01:44:10.568154 24239 raft_consensus.cc:374] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:10.568300 24239 raft_consensus.cc:729] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0901bf3969a04adb8f28852b35c0c350, State: Initialized, Role: FOLLOWER
I20260811 01:44:10.568038 24238 raft_consensus.cc:348] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } }
I20260811 01:44:10.568611 24238 raft_consensus.cc:374] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:10.568760 24238 raft_consensus.cc:729] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01aa5bcc010e4b228017c9f18af9cfa8, State: Initialized, Role: FOLLOWER
I20260811 01:44:10.569103 24239 consensus_queue.cc:260] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } }
I20260811 01:44:10.569474 24238 consensus_queue.cc:260] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } }
I20260811 01:44:10.571482 24239 ts_tablet_manager.cc:1428] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260811 01:44:10.572463 24049 heartbeater.cc:503] Master 127.21.214.126:36965 was elected leader, sending a full tablet report...
I20260811 01:44:10.573446 24238 ts_tablet_manager.cc:1428] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8: Time spent starting tablet: real 0.020s user 0.007s sys 0.013s
I20260811 01:44:10.574016 24245 raft_consensus.cc:482] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:44:10.574447 24245 raft_consensus.cc:504] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } }
I20260811 01:44:10.577034 24245 leader_election.cc:288] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b2781914c654d33a3d8e9228a1c7d1c (127.21.214.67:33957), 01aa5bcc010e4b228017c9f18af9cfa8 (127.21.214.66:43163)
I20260811 01:44:10.591216 24232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:10.578275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:36553 (local address 127.21.214.67:33957)
0811 01:44:10.578574 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:10.578581 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:10.579663 (+ 1082us) server_negotiation.cc:407] Connection header received
0811 01:44:10.579953 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:10.579961 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:10.580054 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:10.580169 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:10.581840 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.583486 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.587866 (+ 4380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.589691 (+ 1825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.589914 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:10.589993 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:10.590804 (+ 811us) server_negotiation.cc:299] Negotiation successful
0811 01:44:10.590956 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 01:44:10.591348 24236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:10.578745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:38465 (local address 127.21.214.66:43163)
0811 01:44:10.578949 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:10.578955 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:10.580601 (+ 1646us) server_negotiation.cc:407] Connection header received
0811 01:44:10.580829 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:10.580833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:10.580894 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:10.581019 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:10.582111 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.583732 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.587890 (+ 4158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.589761 (+ 1871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.589921 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:10.589993 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:10.590810 (+ 817us) server_negotiation.cc:299] Negotiation successful
0811 01:44:10.591094 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 01:44:10.591765 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:10.578003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:43163 (local address 127.21.214.65:38465)
0811 01:44:10.580465 (+ 2462us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:10.580484 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:10.580647 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:10.581044 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:10.581049 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:10.581077 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:10.581904 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:10.581935 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.583880 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.583887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:10.587625 (+ 3738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:10.587643 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.590368 (+ 2725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.590376 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:10.590619 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:10.590658 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 01:44:10.590753 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 01:44:10.590952 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1998,"mutex_wait_us":201,"thread_start_us":1021,"threads_started":1}
I20260811 01:44:10.591899 24247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:10.577451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:33957 (local address 127.21.214.65:36553)
0811 01:44:10.579500 (+ 2049us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:10.579530 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:10.579763 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:10.580465 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:10.580472 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:10.580508 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:10.581630 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:10.581648 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.583640 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.583644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:10.587625 (+ 3981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:10.587643 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.590231 (+ 2588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.590242 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:10.590534 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:10.590589 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 01:44:10.590722 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 01:44:10.590925 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1834,"mutex_wait_us":198,"thread_start_us":2014,"threads_started":1}
I20260811 01:44:10.593614 23991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d32364dfa0e484f96f2585c07f339b0" candidate_uuid: "0901bf3969a04adb8f28852b35c0c350" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" is_pre_election: true
I20260811 01:44:10.593616 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d32364dfa0e484f96f2585c07f339b0" candidate_uuid: "0901bf3969a04adb8f28852b35c0c350" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" is_pre_election: true
I20260811 01:44:10.594362 23991 raft_consensus.cc:2454] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0901bf3969a04adb8f28852b35c0c350 in term 0.
I20260811 01:44:10.594421 24158 raft_consensus.cc:2454] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0901bf3969a04adb8f28852b35c0c350 in term 0.
I20260811 01:44:10.595616 23752 leader_election.cc:302] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 01aa5bcc010e4b228017c9f18af9cfa8, 0901bf3969a04adb8f28852b35c0c350, 4b2781914c654d33a3d8e9228a1c7d1c; no voters:
I20260811 01:44:10.596305 24245 raft_consensus.cc:2781] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:44:10.596549 24245 raft_consensus.cc:482] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:44:10.596783 24245 raft_consensus.cc:3037] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:10.599922 24245 raft_consensus.cc:504] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } }
I20260811 01:44:10.600687 24245 leader_election.cc:288] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [CANDIDATE]: Term 1 election: Requested vote from peers 4b2781914c654d33a3d8e9228a1c7d1c (127.21.214.67:33957), 01aa5bcc010e4b228017c9f18af9cfa8 (127.21.214.66:43163)
I20260811 01:44:10.601388 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d32364dfa0e484f96f2585c07f339b0" candidate_uuid: "0901bf3969a04adb8f28852b35c0c350" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b2781914c654d33a3d8e9228a1c7d1c"
I20260811 01:44:10.601495 23991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d32364dfa0e484f96f2585c07f339b0" candidate_uuid: "0901bf3969a04adb8f28852b35c0c350" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01aa5bcc010e4b228017c9f18af9cfa8"
I20260811 01:44:10.601635 24158 raft_consensus.cc:3037] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:10.601848 23991 raft_consensus.cc:3037] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 0 FOLLOWER]: Advancing to term 1
W20260811 01:44:10.605239 24050 tablet.cc:2369] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:44:10.605608 23991 raft_consensus.cc:2454] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0901bf3969a04adb8f28852b35c0c350 in term 1.
I20260811 01:44:10.605610 24158 raft_consensus.cc:2454] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0901bf3969a04adb8f28852b35c0c350 in term 1.
I20260811 01:44:10.606232 23752 leader_election.cc:302] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01aa5bcc010e4b228017c9f18af9cfa8, 0901bf3969a04adb8f28852b35c0c350; no voters:
I20260811 01:44:10.606652 24245 raft_consensus.cc:2781] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:10.607249 24245 raft_consensus.cc:686] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [term 1 LEADER]: Becoming Leader. State: Replica: 0901bf3969a04adb8f28852b35c0c350, State: Running, Role: LEADER
I20260811 01:44:10.607847 24245 consensus_queue.cc:237] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } }
I20260811 01:44:10.613801 23648 catalog_manager.cc:5495] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0901bf3969a04adb8f28852b35c0c350 (127.21.214.65). New cstate: current_term: 1 leader_uuid: "0901bf3969a04adb8f28852b35c0c350" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } health_report { overall_health: UNKNOWN } } }
I20260811 01:44:10.649372 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:10.642698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55926 (local address 127.21.214.65:34025)
0811 01:44:10.643025 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:10.643033 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:10.643296 (+ 263us) server_negotiation.cc:407] Connection header received
0811 01:44:10.643601 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:10.643605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:10.643685 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:10.643818 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:44:10.644390 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.645330 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.646922 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.647407 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.647506 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:10.648130 (+ 624us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:44:10.648697 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:44:10.648810 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:10.648961 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 01:44:10.649032 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 01:44:10.649931 24216 tablet.cc:2369] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:44:10.659520 24158 raft_consensus.cc:1261] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c [term 1 FOLLOWER]: Refusing update from remote peer 0901bf3969a04adb8f28852b35c0c350: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:10.659543 23991 raft_consensus.cc:1261] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 1 FOLLOWER]: Refusing update from remote peer 0901bf3969a04adb8f28852b35c0c350: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:10.660713 24249 consensus_queue.cc:1035] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:10.661126 24245 consensus_queue.cc:1035] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:10.678893 24252 mvcc.cc:205] Tried to move back new op lower bound from 7317146217090150400 to 7317146216893771776. Current Snapshot: MvccSnapshot[applied={T|T < 7317146217090150400}]
I20260811 01:44:10.684191 24253 mvcc.cc:205] Tried to move back new op lower bound from 7317146217090150400 to 7317146216893771776. Current Snapshot: MvccSnapshot[applied={T|T < 7317146217090150400}]
I20260811 01:44:10.706327 23648 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:44:10.706837 23648 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:44:10.711596 23648 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3: Sending DeleteTablet for 3 replicas of tablet 2d32364dfa0e484f96f2585c07f339b0
I20260811 01:44:10.713343 23962 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d32364dfa0e484f96f2585c07f339b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:10 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:40900
I20260811 01:44:10.713296 24132 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d32364dfa0e484f96f2585c07f339b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:10 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:34366
I20260811 01:44:10.714248 23805 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d32364dfa0e484f96f2585c07f339b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:10 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:55920
I20260811 01:44:10.716531 23648 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:44:10.717803 23648 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:44:10.722481 24262 tablet_replica.cc:330] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c: stopping tablet replica
I20260811 01:44:10.729575 24132 tablet_service.cc:1467] Processing CreateTablet for tablet 7e50eda1c18b4069ac15ce6690682f92 (DEFAULT_TABLE table=test-table [id=2d4590335a5d43688f542fcd4f62708c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:10.729588 23961 tablet_service.cc:1467] Processing CreateTablet for tablet 7e50eda1c18b4069ac15ce6690682f92 (DEFAULT_TABLE table=test-table [id=2d4590335a5d43688f542fcd4f62708c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:10.730422 24132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e50eda1c18b4069ac15ce6690682f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:10.730475 23804 tablet_service.cc:1467] Processing CreateTablet for tablet 7e50eda1c18b4069ac15ce6690682f92 (DEFAULT_TABLE table=test-table [id=2d4590335a5d43688f542fcd4f62708c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:10.730785 23961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e50eda1c18b4069ac15ce6690682f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:10.731390 23804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e50eda1c18b4069ac15ce6690682f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:10.736188 24262 raft_consensus.cc:2229] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:44:10.737188 24262 raft_consensus.cc:2258] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:10.738914 24263 tablet_replica.cc:330] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350: stopping tablet replica
I20260811 01:44:10.739786 24263 raft_consensus.cc:2229] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:44:10.741318 24263 raft_consensus.cc:2258] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:10.741617 24264 tablet_replica.cc:330] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8: stopping tablet replica
I20260811 01:44:10.742090 24264 raft_consensus.cc:2229] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:44:10.742664 24264 raft_consensus.cc:2258] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:10.743479 24239 tablet_bootstrap.cc:492] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350: Bootstrap starting.
I20260811 01:44:10.744136 24262 ts_tablet_manager.cc:1905] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:10.745239 24238 tablet_bootstrap.cc:492] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8: Bootstrap starting.
I20260811 01:44:10.745921 24237 tablet_bootstrap.cc:492] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c: Bootstrap starting.
I20260811 01:44:10.747861 24238 tablet_bootstrap.cc:654] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:10.751284 24237 tablet_bootstrap.cc:654] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:10.751796 24263 ts_tablet_manager.cc:1905] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:10.753094 24239 tablet_bootstrap.cc:654] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:10.756633 24264 ts_tablet_manager.cc:1905] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:10.763247 24238 tablet_bootstrap.cc:492] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8: No bootstrap required, opened a new log
I20260811 01:44:10.763617 24238 ts_tablet_manager.cc:1397] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.007s
I20260811 01:44:10.765020 24238 raft_consensus.cc:348] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } }
I20260811 01:44:10.765514 24238 raft_consensus.cc:374] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:10.765769 24238 raft_consensus.cc:729] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01aa5bcc010e4b228017c9f18af9cfa8, State: Initialized, Role: FOLLOWER
I20260811 01:44:10.766214 24238 consensus_queue.cc:260] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } }
I20260811 01:44:10.768150 24238 ts_tablet_manager.cc:1428] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 01:44:10.770300 24237 tablet_bootstrap.cc:492] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c: No bootstrap required, opened a new log
I20260811 01:44:10.770543 24237 ts_tablet_manager.cc:1397] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.005s
I20260811 01:44:10.771917 24237 raft_consensus.cc:348] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } }
I20260811 01:44:10.772194 24237 raft_consensus.cc:374] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:10.772418 24237 raft_consensus.cc:729] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b2781914c654d33a3d8e9228a1c7d1c, State: Initialized, Role: FOLLOWER
I20260811 01:44:10.773042 24263 ts_tablet_manager.cc:1918] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:44:10.775719 24264 ts_tablet_manager.cc:1918] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:44:10.776051 24264 log.cc:1196] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/2d32364dfa0e484f96f2585c07f339b0
I20260811 01:44:10.774462 24239 tablet_bootstrap.cc:492] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350: No bootstrap required, opened a new log
I20260811 01:44:10.776480 24239 ts_tablet_manager.cc:1397] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350: Time spent bootstrapping tablet: real 0.033s user 0.004s sys 0.004s
I20260811 01:44:10.777179 24237 consensus_queue.cc:260] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } }
I20260811 01:44:10.777989 24264 ts_tablet_manager.cc:1939] T 2d32364dfa0e484f96f2585c07f339b0 P 01aa5bcc010e4b228017c9f18af9cfa8: Deleting consensus metadata
I20260811 01:44:10.778117 24263 log.cc:1196] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/2d32364dfa0e484f96f2585c07f339b0
I20260811 01:44:10.779083 24263 ts_tablet_manager.cc:1939] T 2d32364dfa0e484f96f2585c07f339b0 P 0901bf3969a04adb8f28852b35c0c350: Deleting consensus metadata
I20260811 01:44:10.779357 24239 raft_consensus.cc:348] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } }
I20260811 01:44:10.779989 24239 raft_consensus.cc:374] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:10.780542 24239 raft_consensus.cc:729] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0901bf3969a04adb8f28852b35c0c350, State: Initialized, Role: FOLLOWER
I20260811 01:44:10.781056 24239 consensus_queue.cc:260] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } }
I20260811 01:44:10.781093 23633 catalog_manager.cc:4841] TS 01aa5bcc010e4b228017c9f18af9cfa8 (127.21.214.66:43163): tablet 2d32364dfa0e484f96f2585c07f339b0 (table test-table [id=793f414ae6bc454b8341f02b0bc5b1cf]) successfully deleted
I20260811 01:44:10.781304 24237 ts_tablet_manager.cc:1428] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260811 01:44:10.783236 23634 catalog_manager.cc:4841] TS 0901bf3969a04adb8f28852b35c0c350 (127.21.214.65:34025): tablet 2d32364dfa0e484f96f2585c07f339b0 (table test-table [id=793f414ae6bc454b8341f02b0bc5b1cf]) successfully deleted
I20260811 01:44:10.784022 24239 ts_tablet_manager.cc:1428] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 01:44:10.787521 24262 ts_tablet_manager.cc:1918] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:44:10.787874 24262 log.cc:1196] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/2d32364dfa0e484f96f2585c07f339b0
I20260811 01:44:10.788751 24262 ts_tablet_manager.cc:1939] T 2d32364dfa0e484f96f2585c07f339b0 P 4b2781914c654d33a3d8e9228a1c7d1c: Deleting consensus metadata
I20260811 01:44:10.791515 23632 catalog_manager.cc:4841] TS 4b2781914c654d33a3d8e9228a1c7d1c (127.21.214.67:33957): tablet 2d32364dfa0e484f96f2585c07f339b0 (table test-table [id=793f414ae6bc454b8341f02b0bc5b1cf]) successfully deleted
W20260811 01:44:10.802727 23886 tablet.cc:2369] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:44:10.824628 24245 raft_consensus.cc:482] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:44:10.824934 24245 raft_consensus.cc:504] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } }
I20260811 01:44:10.826143 24245 leader_election.cc:288] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01aa5bcc010e4b228017c9f18af9cfa8 (127.21.214.66:43163), 4b2781914c654d33a3d8e9228a1c7d1c (127.21.214.67:33957)
I20260811 01:44:10.826686 23991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e50eda1c18b4069ac15ce6690682f92" candidate_uuid: "0901bf3969a04adb8f28852b35c0c350" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" is_pre_election: true
I20260811 01:44:10.826854 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e50eda1c18b4069ac15ce6690682f92" candidate_uuid: "0901bf3969a04adb8f28852b35c0c350" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" is_pre_election: true
I20260811 01:44:10.827054 23991 raft_consensus.cc:2454] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0901bf3969a04adb8f28852b35c0c350 in term 0.
I20260811 01:44:10.827239 24158 raft_consensus.cc:2454] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0901bf3969a04adb8f28852b35c0c350 in term 0.
I20260811 01:44:10.827589 23752 leader_election.cc:302] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01aa5bcc010e4b228017c9f18af9cfa8, 0901bf3969a04adb8f28852b35c0c350; no voters:
I20260811 01:44:10.828071 24245 raft_consensus.cc:2781] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:44:10.828246 24245 raft_consensus.cc:482] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:44:10.828367 24245 raft_consensus.cc:3037] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:10.831837 24245 raft_consensus.cc:504] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } }
I20260811 01:44:10.833390 23991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e50eda1c18b4069ac15ce6690682f92" candidate_uuid: "0901bf3969a04adb8f28852b35c0c350" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01aa5bcc010e4b228017c9f18af9cfa8"
I20260811 01:44:10.833511 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e50eda1c18b4069ac15ce6690682f92" candidate_uuid: "0901bf3969a04adb8f28852b35c0c350" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b2781914c654d33a3d8e9228a1c7d1c"
I20260811 01:44:10.833757 23991 raft_consensus.cc:3037] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:10.833932 24158 raft_consensus.cc:3037] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:10.836580 23991 raft_consensus.cc:2454] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0901bf3969a04adb8f28852b35c0c350 in term 1.
I20260811 01:44:10.836771 24245 leader_election.cc:288] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [CANDIDATE]: Term 1 election: Requested vote from peers 01aa5bcc010e4b228017c9f18af9cfa8 (127.21.214.66:43163), 4b2781914c654d33a3d8e9228a1c7d1c (127.21.214.67:33957)
I20260811 01:44:10.837113 23752 leader_election.cc:302] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01aa5bcc010e4b228017c9f18af9cfa8, 0901bf3969a04adb8f28852b35c0c350; no voters:
I20260811 01:44:10.837575 24245 raft_consensus.cc:2781] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:10.837822 24245 raft_consensus.cc:686] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [term 1 LEADER]: Becoming Leader. State: Replica: 0901bf3969a04adb8f28852b35c0c350, State: Running, Role: LEADER
I20260811 01:44:10.838287 24245 consensus_queue.cc:237] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } }
I20260811 01:44:10.838549 24158 raft_consensus.cc:2454] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0901bf3969a04adb8f28852b35c0c350 in term 1.
I20260811 01:44:10.842365 23647 catalog_manager.cc:5495] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0901bf3969a04adb8f28852b35c0c350 (127.21.214.65). New cstate: current_term: 1 leader_uuid: "0901bf3969a04adb8f28852b35c0c350" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } health_report { overall_health: UNKNOWN } } }
Aug 11 01:44:10 dist-test-slave-9gcw krb5kdc[23600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412648, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:10.930816 24275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:10.908970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36965 (local address 127.0.0.1:33756)
0811 01:44:10.910362 (+ 1392us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:10.910381 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:10.910517 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:10.910932 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:10.910935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:10.911888 (+ 953us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:10.912367 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:10.912379 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.914013 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.914017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:10.915597 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:10.915611 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.916915 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.916920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:10.917090 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:10.918053 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:10.918102 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:10.921088 (+ 2986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:10.926527 (+ 5439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:10.926563 (+ 36us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:10.926576 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:10.927403 (+ 827us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:10.927999 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:10.928007 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:10.928014 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:10.928341 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:10.929493 (+ 1152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:10.929499 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:10.929856 (+ 357us) client_negotiation.cc:769] Sending connection context
0811 01:44:10.930106 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 01:44:10.930376 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1128,"thread_start_us":839,"threads_started":1}
I20260811 01:44:10.932039 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:10.908984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33756 (local address 127.21.214.126:36965)
0811 01:44:10.909353 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:10.909361 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:10.910489 (+ 1128us) server_negotiation.cc:407] Connection header received
0811 01:44:10.910645 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:10.910652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:10.910763 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:10.910906 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:10.912549 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.913821 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.915753 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:10.916669 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:10.916827 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:10.921435 (+ 4608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:10.921447 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:10.921456 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:10.921519 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:10.926344 (+ 4825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:10.927555 (+ 1211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:10.927611 (+ 56us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:10.927615 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:10.927858 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:10.928482 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:10.928490 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:10.928497 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:10.929184 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:10.929434 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:10.931387 (+ 1953us) server_negotiation.cc:299] Negotiation successful
0811 01:44:10.931701 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 01:44:10.939419 23647 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:33756:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 01:44:10.940970 23647 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:44:10.950091 23961 tablet_service.cc:1467] Processing CreateTablet for tablet c57188652f634c10b5f7af08598b8ed2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ee654ac759784a5ab127e9f04d247911]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 01:44:10.950994 23961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c57188652f634c10b5f7af08598b8ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:10.950886 24132 tablet_service.cc:1467] Processing CreateTablet for tablet c57188652f634c10b5f7af08598b8ed2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ee654ac759784a5ab127e9f04d247911]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 01:44:10.951567 24132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c57188652f634c10b5f7af08598b8ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:10.953301 23804 tablet_service.cc:1467] Processing CreateTablet for tablet c57188652f634c10b5f7af08598b8ed2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ee654ac759784a5ab127e9f04d247911]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 01:44:10.953931 23804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c57188652f634c10b5f7af08598b8ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:10.961057 24238 tablet_bootstrap.cc:492] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8: Bootstrap starting.
I20260811 01:44:10.962771 24237 tablet_bootstrap.cc:492] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c: Bootstrap starting.
I20260811 01:44:10.963894 24238 tablet_bootstrap.cc:654] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:10.965598 24237 tablet_bootstrap.cc:654] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:10.967226 24239 tablet_bootstrap.cc:492] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350: Bootstrap starting.
I20260811 01:44:10.969617 24237 tablet_bootstrap.cc:492] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c: No bootstrap required, opened a new log
I20260811 01:44:10.969872 24237 ts_tablet_manager.cc:1397] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260811 01:44:10.970713 24239 tablet_bootstrap.cc:654] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:10.970712 24237 raft_consensus.cc:348] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } }
I20260811 01:44:10.970997 24237 raft_consensus.cc:374] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:10.971131 24237 raft_consensus.cc:729] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b2781914c654d33a3d8e9228a1c7d1c, State: Initialized, Role: FOLLOWER
I20260811 01:44:10.971535 24237 consensus_queue.cc:260] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } }
I20260811 01:44:10.972383 24237 ts_tablet_manager.cc:1428] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 01:44:10.972288 24244 tablet_replica.cc:439] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } }
I20260811 01:44:10.972761 24244 tablet_replica.cc:442] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:44:10.976976 24238 tablet_bootstrap.cc:492] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8: No bootstrap required, opened a new log
I20260811 01:44:10.977135 24238 ts_tablet_manager.cc:1397] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.003s
I20260811 01:44:10.978164 24238 raft_consensus.cc:348] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } }
I20260811 01:44:10.978338 24239 tablet_bootstrap.cc:492] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350: No bootstrap required, opened a new log
I20260811 01:44:10.978511 24238 raft_consensus.cc:374] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:10.978617 24239 ts_tablet_manager.cc:1397] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260811 01:44:10.978682 24238 raft_consensus.cc:729] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01aa5bcc010e4b228017c9f18af9cfa8, State: Initialized, Role: FOLLOWER
I20260811 01:44:10.979125 24238 consensus_queue.cc:260] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } }
I20260811 01:44:10.979787 24239 raft_consensus.cc:348] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } }
I20260811 01:44:10.979907 24246 tablet_replica.cc:439] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } }
I20260811 01:44:10.980337 24239 raft_consensus.cc:374] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:10.980459 24246 tablet_replica.cc:442] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:44:10.980510 24239 raft_consensus.cc:729] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0901bf3969a04adb8f28852b35c0c350, State: Initialized, Role: FOLLOWER
I20260811 01:44:10.981171 24238 ts_tablet_manager.cc:1428] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 01:44:10.980935 24239 consensus_queue.cc:260] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } }
I20260811 01:44:10.981663 24245 tablet_replica.cc:439] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } }
I20260811 01:44:10.982313 24245 tablet_replica.cc:442] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:44:10.983755 24239 ts_tablet_manager.cc:1428] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260811 01:44:11.067840 24245 raft_consensus.cc:482] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:44:11.068261 24245 raft_consensus.cc:504] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } }
I20260811 01:44:11.069136 24245 leader_election.cc:288] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01aa5bcc010e4b228017c9f18af9cfa8 (127.21.214.66:43163), 4b2781914c654d33a3d8e9228a1c7d1c (127.21.214.67:33957)
I20260811 01:44:11.069736 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c57188652f634c10b5f7af08598b8ed2" candidate_uuid: "0901bf3969a04adb8f28852b35c0c350" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" is_pre_election: true
I20260811 01:44:11.069852 23991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c57188652f634c10b5f7af08598b8ed2" candidate_uuid: "0901bf3969a04adb8f28852b35c0c350" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" is_pre_election: true
I20260811 01:44:11.070046 24158 raft_consensus.cc:2454] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0901bf3969a04adb8f28852b35c0c350 in term 0.
I20260811 01:44:11.070374 23991 raft_consensus.cc:2454] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0901bf3969a04adb8f28852b35c0c350 in term 0.
I20260811 01:44:11.070596 23751 leader_election.cc:302] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0901bf3969a04adb8f28852b35c0c350, 4b2781914c654d33a3d8e9228a1c7d1c; no voters:
I20260811 01:44:11.071105 24245 raft_consensus.cc:2781] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:44:11.071290 24245 raft_consensus.cc:482] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:44:11.071542 24245 raft_consensus.cc:3037] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:11.074277 24245 raft_consensus.cc:504] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } }
I20260811 01:44:11.075119 24245 leader_election.cc:288] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [CANDIDATE]: Term 1 election: Requested vote from peers 01aa5bcc010e4b228017c9f18af9cfa8 (127.21.214.66:43163), 4b2781914c654d33a3d8e9228a1c7d1c (127.21.214.67:33957)
I20260811 01:44:11.075593 23991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c57188652f634c10b5f7af08598b8ed2" candidate_uuid: "0901bf3969a04adb8f28852b35c0c350" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01aa5bcc010e4b228017c9f18af9cfa8"
I20260811 01:44:11.075704 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c57188652f634c10b5f7af08598b8ed2" candidate_uuid: "0901bf3969a04adb8f28852b35c0c350" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b2781914c654d33a3d8e9228a1c7d1c"
I20260811 01:44:11.075928 23991 raft_consensus.cc:3037] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:11.076081 24158 raft_consensus.cc:3037] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:11.079274 24158 raft_consensus.cc:2454] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0901bf3969a04adb8f28852b35c0c350 in term 1.
I20260811 01:44:11.079854 23751 leader_election.cc:302] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0901bf3969a04adb8f28852b35c0c350, 4b2781914c654d33a3d8e9228a1c7d1c; no voters:
I20260811 01:44:11.080175 24245 raft_consensus.cc:2781] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:11.080487 24245 raft_consensus.cc:686] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [term 1 LEADER]: Becoming Leader. State: Replica: 0901bf3969a04adb8f28852b35c0c350, State: Running, Role: LEADER
I20260811 01:44:11.080492 23991 raft_consensus.cc:2454] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0901bf3969a04adb8f28852b35c0c350 in term 1.
I20260811 01:44:11.080801 24245 consensus_queue.cc:237] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } }
I20260811 01:44:11.084059 24257 tablet_replica.cc:439] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350: TxnStatusTablet state changed. Reason: New leader 0901bf3969a04adb8f28852b35c0c350. Latest consensus state: current_term: 1 leader_uuid: "0901bf3969a04adb8f28852b35c0c350" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } }
I20260811 01:44:11.084774 24257 tablet_replica.cc:442] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350: This TxnStatusTablet replica's current role is: LEADER
I20260811 01:44:11.085387 23648 catalog_manager.cc:5495] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0901bf3969a04adb8f28852b35c0c350 (127.21.214.65). New cstate: current_term: 1 leader_uuid: "0901bf3969a04adb8f28852b35c0c350" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } health_report { overall_health: UNKNOWN } } }
I20260811 01:44:11.086673 24280 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 01:44:11.087111 24280 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 01:44:11.137773 24275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:11.127878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:34025 (local address 127.0.0.1:55934)
0811 01:44:11.128145 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:11.128177 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:11.128353 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:11.128817 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:11.128824 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:11.128839 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 01:44:11.129062 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:11.129073 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:11.131468 (+ 2395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:11.131477 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:11.134580 (+ 3103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:11.134605 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:11.136051 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:11.136060 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:11.136385 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:11.136507 (+ 122us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 01:44:11.137184 (+ 677us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 01:44:11.137212 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 01:44:11.137302 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 01:44:11.137491 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 01:44:11.138482 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:11.127584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55934 (local address 127.21.214.65:34025)
0811 01:44:11.127790 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:11.127795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:11.128274 (+ 479us) server_negotiation.cc:407] Connection header received
0811 01:44:11.128563 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:11.128568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:11.128652 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:11.128764 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:44:11.129370 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:11.131221 (+ 1851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:11.134894 (+ 3673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:11.135721 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:11.135873 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:11.136747 (+ 874us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:44:11.136983 (+ 236us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:44:11.137099 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:11.138053 (+ 954us) server_negotiation.cc:299] Negotiation successful
0811 01:44:11.138153 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 01:44:11.144284 23991 raft_consensus.cc:1261] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 1 FOLLOWER]: Refusing update from remote peer 0901bf3969a04adb8f28852b35c0c350: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:11.144321 24158 raft_consensus.cc:1261] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c [term 1 FOLLOWER]: Refusing update from remote peer 0901bf3969a04adb8f28852b35c0c350: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:11.145166 24249 consensus_queue.cc:1035] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:11.145535 24257 consensus_queue.cc:1035] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:11.158398 24244 tablet_replica.cc:439] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c: TxnStatusTablet state changed. Reason: New leader 0901bf3969a04adb8f28852b35c0c350. Latest consensus state: current_term: 1 leader_uuid: "0901bf3969a04adb8f28852b35c0c350" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } }
I20260811 01:44:11.158854 24244 tablet_replica.cc:442] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:44:11.165246 24249 tablet_replica.cc:439] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0901bf3969a04adb8f28852b35c0c350" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } }
I20260811 01:44:11.165812 24249 tablet_replica.cc:442] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350: This TxnStatusTablet replica's current role is: LEADER
I20260811 01:44:11.170504 24254 mvcc.cc:205] Tried to move back new op lower bound from 7317146219078070272 to 7317146218830417920. Current Snapshot: MvccSnapshot[applied={T|T < 7317146219078070272}]
I20260811 01:44:11.170569 24246 tablet_replica.cc:439] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8: TxnStatusTablet state changed. Reason: New leader 0901bf3969a04adb8f28852b35c0c350. Latest consensus state: current_term: 1 leader_uuid: "0901bf3969a04adb8f28852b35c0c350" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } }
I20260811 01:44:11.170885 24246 tablet_replica.cc:442] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:44:11.170929 24244 tablet_replica.cc:439] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0901bf3969a04adb8f28852b35c0c350" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } }
I20260811 01:44:11.171187 24244 tablet_replica.cc:442] T c57188652f634c10b5f7af08598b8ed2 P 4b2781914c654d33a3d8e9228a1c7d1c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:44:11.173632 24249 tablet_replica.cc:439] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0901bf3969a04adb8f28852b35c0c350" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } }
I20260811 01:44:11.173980 24249 tablet_replica.cc:442] T c57188652f634c10b5f7af08598b8ed2 P 0901bf3969a04adb8f28852b35c0c350: This TxnStatusTablet replica's current role is: LEADER
I20260811 01:44:11.175978 24246 tablet_replica.cc:439] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0901bf3969a04adb8f28852b35c0c350" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0901bf3969a04adb8f28852b35c0c350" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34025 } } peers { permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 } } peers { permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 } } }
I20260811 01:44:11.176223 24246 tablet_replica.cc:442] T c57188652f634c10b5f7af08598b8ed2 P 01aa5bcc010e4b228017c9f18af9cfa8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:44:11.198573 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:11.190385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:34025 (local address 127.21.214.65:35479)
0811 01:44:11.191169 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:11.191186 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:11.191319 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:11.191718 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:11.191721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:11.191734 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 01:44:11.192074 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:11.192083 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:11.194050 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:11.194054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:11.195450 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:11.195467 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:11.196991 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:11.196996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:11.197204 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:11.197290 (+ 86us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 01:44:11.197948 (+ 658us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 01:44:11.197957 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 01:44:11.198027 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 01:44:11.198149 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":412,"threads_started":1}
I20260811 01:44:11.198900 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:11.190546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:35479 (local address 127.21.214.65:34025)
0811 01:44:11.190820 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:11.190828 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:11.191297 (+ 469us) server_negotiation.cc:407] Connection header received
0811 01:44:11.191474 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:11.191480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:11.191581 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:11.191713 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:44:11.192242 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:11.193813 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:11.195626 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:11.196700 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:11.196939 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:11.197551 (+ 612us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:44:11.197806 (+ 255us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:44:11.197955 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:11.198086 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 01:44:11.198217 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":104}
I20260811 01:44:11.210547 23991 raft_consensus.cc:1261] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 1 FOLLOWER]: Refusing update from remote peer 0901bf3969a04adb8f28852b35c0c350: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:11.211488 24158 raft_consensus.cc:1261] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c [term 1 FOLLOWER]: Refusing update from remote peer 0901bf3969a04adb8f28852b35c0c350: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:11.211402 24245 consensus_queue.cc:1035] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01aa5bcc010e4b228017c9f18af9cfa8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:11.212181 24245 consensus_queue.cc:1035] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b2781914c654d33a3d8e9228a1c7d1c" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 33957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:11.296272 23648 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:44:11.296500 23648 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:44:11.300246 23648 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5598d5d440d4861b80cbb6b3a5721f3: Sending DeleteTablet for 3 replicas of tablet 7e50eda1c18b4069ac15ce6690682f92
I20260811 01:44:11.301316 23805 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e50eda1c18b4069ac15ce6690682f92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:11 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:55920
I20260811 01:44:11.301625 24132 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e50eda1c18b4069ac15ce6690682f92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:11 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:34366
I20260811 01:44:11.303256 23961 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e50eda1c18b4069ac15ce6690682f92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:11 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:40900
I20260811 01:44:11.310667 24294 tablet_replica.cc:330] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c: stopping tablet replica
I20260811 01:44:11.311177 24293 tablet_replica.cc:330] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350: stopping tablet replica
I20260811 01:44:11.311159 24294 raft_consensus.cc:2229] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:44:11.311408 24293 raft_consensus.cc:2229] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:44:11.311776 24294 raft_consensus.cc:2258] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:11.312500 24293 raft_consensus.cc:2258] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:11.313452 24294 ts_tablet_manager.cc:1905] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:11.313750 24293 ts_tablet_manager.cc:1905] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:11.323211 24294 ts_tablet_manager.cc:1918] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 01:44:11.323436 24294 log.cc:1196] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/7e50eda1c18b4069ac15ce6690682f92
I20260811 01:44:11.324061 24294 ts_tablet_manager.cc:1939] T 7e50eda1c18b4069ac15ce6690682f92 P 4b2781914c654d33a3d8e9228a1c7d1c: Deleting consensus metadata
I20260811 01:44:11.325529 23632 catalog_manager.cc:4841] TS 4b2781914c654d33a3d8e9228a1c7d1c (127.21.214.67:33957): tablet 7e50eda1c18b4069ac15ce6690682f92 (table test-table [id=2d4590335a5d43688f542fcd4f62708c]) successfully deleted
I20260811 01:44:11.326011 24296 tablet_replica.cc:330] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8: stopping tablet replica
I20260811 01:44:11.326306 24296 raft_consensus.cc:2229] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:44:11.326563 24296 raft_consensus.cc:2258] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:11.328709 24296 ts_tablet_manager.cc:1905] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:11.329905 24293 ts_tablet_manager.cc:1918] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 01:44:11.330135 24293 log.cc:1196] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/7e50eda1c18b4069ac15ce6690682f92
I20260811 01:44:11.330933 24293 ts_tablet_manager.cc:1939] T 7e50eda1c18b4069ac15ce6690682f92 P 0901bf3969a04adb8f28852b35c0c350: Deleting consensus metadata
I20260811 01:44:11.332366 23634 catalog_manager.cc:4841] TS 0901bf3969a04adb8f28852b35c0c350 (127.21.214.65:34025): tablet 7e50eda1c18b4069ac15ce6690682f92 (table test-table [id=2d4590335a5d43688f542fcd4f62708c]) successfully deleted
Aug 11 01:44:11 dist-test-slave-9gcw krb5kdc[23600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412650, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:44:11.341717 24296 ts_tablet_manager.cc:1918] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 01:44:11.342459 24296 log.cc:1196] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/7e50eda1c18b4069ac15ce6690682f92
I20260811 01:44:11.343241 24296 ts_tablet_manager.cc:1939] T 7e50eda1c18b4069ac15ce6690682f92 P 01aa5bcc010e4b228017c9f18af9cfa8: Deleting consensus metadata
I20260811 01:44:11.344767 23633 catalog_manager.cc:4841] TS 01aa5bcc010e4b228017c9f18af9cfa8 (127.21.214.66:43163): tablet 7e50eda1c18b4069ac15ce6690682f92 (table test-table [id=2d4590335a5d43688f542fcd4f62708c]) successfully deleted
I20260811 01:44:11.349774 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:11.314713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55946 (local address 127.21.214.65:34025)
0811 01:44:11.314921 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:11.314928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:11.319278 (+ 4350us) server_negotiation.cc:407] Connection header received
0811 01:44:11.321216 (+ 1938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:11.321224 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:11.321318 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:11.321494 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:11.325133 (+ 3639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:11.327371 (+ 2238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:11.330483 (+ 3112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:11.331508 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:11.331624 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:11.339082 (+ 7458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:11.339106 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:11.339124 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:11.339163 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:11.345300 (+ 6137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:11.346302 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:11.346314 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:11.346322 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:11.346497 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:11.346985 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:11.346990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:11.346993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:11.347674 (+ 681us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:11.348145 (+ 471us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:11.348452 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 01:44:11.348705 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 01:44:11.350788 23880 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55946
I20260811 01:44:11.377790 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:11.354411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33770 (local address 127.21.214.126:36965)
0811 01:44:11.354681 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:11.354690 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:11.361195 (+ 6505us) server_negotiation.cc:407] Connection header received
0811 01:44:11.361520 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:11.361531 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:11.361604 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:11.361727 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:11.363402 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:11.364216 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:11.365874 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:11.366502 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:11.366649 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:11.370326 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:11.370333 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:11.370336 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:11.370370 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:11.374038 (+ 3668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:11.374873 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:11.374877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:11.374880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:11.375020 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:11.375561 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:11.375565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:11.375567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:11.375850 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:11.376275 (+ 425us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:11.377215 (+ 940us) server_negotiation.cc:299] Negotiation successful
0811 01:44:11.377461 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260811 01:44:11.378520 23648 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33770
I20260811 01:44:11.380915 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 23734
I20260811 01:44:11.398448 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:11.381821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55952 (local address 127.21.214.65:34025)
0811 01:44:11.382304 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:11.382311 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:11.382496 (+ 185us) server_negotiation.cc:407] Connection header received
0811 01:44:11.382737 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:11.382742 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:11.382817 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:11.382927 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:11.384171 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:11.385095 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:11.386521 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:11.387162 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:11.387324 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:11.390627 (+ 3303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:11.390634 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:11.390638 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:11.390672 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:11.394785 (+ 4113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:11.395724 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:11.395737 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:11.395741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:11.396011 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:11.396453 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:11.396458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:11.396461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:11.396888 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:11.397222 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:11.397533 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 01:44:11.397874 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327}
W20260811 01:44:11.399200 23880 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55952
W20260811 01:44:11.399855 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 23734: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:44:11.400053 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 23734
I20260811 01:44:11.429072 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 23889
Aug 11 01:44:11 dist-test-slave-9gcw krb5kdc[23600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412650, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.214.66@KRBTEST.COM
I20260811 01:44:11.445096 24302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:11.429962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40906 (local address 127.21.214.66:43163)
0811 01:44:11.430458 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:11.430462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:11.430477 (+ 15us) server_negotiation.cc:407] Connection header received
0811 01:44:11.430533 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:11.430536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:11.430595 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:11.430719 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:11.431972 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:11.432760 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:11.434284 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:11.434709 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:11.434849 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:11.439025 (+ 4176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:11.439040 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:11.439057 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:11.439098 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:11.442228 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:11.442942 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:11.442951 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:11.442953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:11.443092 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:11.443486 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:11.443491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:11.443494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:11.443862 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:11.444162 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:11.444464 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 01:44:11.444701 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":243,"threads_started":1}
W20260811 01:44:11.445910 24044 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40906
W20260811 01:44:11.446601 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 23889: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:44:11.446768 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 23889
I20260811 01:44:11.471933 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 24054
Aug 11 01:44:11 dist-test-slave-9gcw krb5kdc[23600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412650, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.214.67@KRBTEST.COM
I20260811 01:44:11.495620 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:11.472804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34376 (local address 127.21.214.67:33957)
0811 01:44:11.473328 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:11.473334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:11.473352 (+ 18us) server_negotiation.cc:407] Connection header received
0811 01:44:11.473409 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:11.473413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:11.473484 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:11.473587 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:11.474866 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:11.475792 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:11.477278 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:11.477855 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:11.477958 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:11.487918 (+ 9960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:11.487937 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:11.487956 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:11.487998 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:11.491685 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:11.492726 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:11.492735 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:11.492737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:11.492945 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:11.493520 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:11.493526 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:11.493528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:11.494033 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:11.494420 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:11.494808 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 01:44:11.495168 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":257,"threads_started":1}
W20260811 01:44:11.496485 24210 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34376
W20260811 01:44:11.497336 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 24054: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:44:11.497535 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 24054
I20260811 01:44:11.520676 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 23614
I20260811 01:44:11.521982 23693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:11.642324 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 23614
2026-08-11T01:44:11Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4946 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:44:11 dist-test-slave-9gcw krb5kdc[24307](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:44:11 dist-test-slave-9gcw krb5kdc[24307](info): set up 2 sockets
Aug 11 01:44:11 dist-test-slave-9gcw krb5kdc[24307](info): commencing operation
krb5kdc: starting...
W20260811 01:44:12.880568 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:44:13 dist-test-slave-9gcw krb5kdc[24307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:44:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:44:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:44:13.113838 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:40057
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:40057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:13.479596 24324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:13.479988 24324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:13.480170 24324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:13.487778 24324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:44:13.487967 24324 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 01:44:13.488073 24324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:13.488209 24324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:13.488307 24324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:44:13.488410 24324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:44:13.499065 24324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:40057
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:40057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:13.500578 24324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:13.502080 24324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:13.514735 24329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:13.514797 24332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:13.514825 24330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:13.518936 24324 server_base.cc:1034] running on GCE node
I20260811 01:44:13.519902 24324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:13.521549 24324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:13.522820 24324 hybrid_clock.cc:648] HybridClock initialized: now 1786412653522758 us; error 44 us; skew 500 ppm
I20260811 01:44:13.523177 24324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:13 dist-test-slave-9gcw krb5kdc[24307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412653, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:13.526957 24324 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:44:13.528680 24324 webserver.cc:460] Webserver started at http://127.21.214.126:36329/ using document root <none> and password file <none>
I20260811 01:44:13.529565 24324 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:13.529670 24324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:13.529981 24324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:13.533459 24324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "926939ec8ecc414db40558db089b1268"
format_stamp: "Formatted at 2026-08-11 01:44:13 on dist-test-slave-9gcw"
server_key: "c572b8faa9f3a72810223d7729634a3d"
server_key_iv: "af755d686d27a1cf80d702e5a8ffac25"
server_key_version: "encryptionkey@0"
I20260811 01:44:13.534288 24324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "926939ec8ecc414db40558db089b1268"
format_stamp: "Formatted at 2026-08-11 01:44:13 on dist-test-slave-9gcw"
server_key: "c572b8faa9f3a72810223d7729634a3d"
server_key_iv: "af755d686d27a1cf80d702e5a8ffac25"
server_key_version: "encryptionkey@0"
I20260811 01:44:13.540256 24324 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 01:44:13.544203 24340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:13.545830 24324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:44:13.546039 24324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "926939ec8ecc414db40558db089b1268"
format_stamp: "Formatted at 2026-08-11 01:44:13 on dist-test-slave-9gcw"
server_key: "c572b8faa9f3a72810223d7729634a3d"
server_key_iv: "af755d686d27a1cf80d702e5a8ffac25"
server_key_version: "encryptionkey@0"
I20260811 01:44:13.546314 24324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:13.563519 24324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:13.568642 24324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:13.568969 24324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:13.593218 24324 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:40057
I20260811 01:44:13.593202 24412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:40057 every 8 connection(s)
I20260811 01:44:13.594949 24324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:44:13.597401 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 24324
I20260811 01:44:13.597647 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:44:13.598356 22361 external_mini_cluster.cc:1428] Setting key ef5892d083d98d023a08175d03496017
I20260811 01:44:13.599218 24413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:13.614702 24413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268: Bootstrap starting.
Aug 11 01:44:13 dist-test-slave-9gcw krb5kdc[24307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:13.618746 24413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:13.620067 24413 log.cc:824] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:13.623283 24413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268: No bootstrap required, opened a new log
I20260811 01:44:13.625233 24417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:13.600720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49468 (local address 127.21.214.126:40057)
0811 01:44:13.601645 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:13.601658 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:13.601704 (+ 46us) server_negotiation.cc:407] Connection header received
0811 01:44:13.604397 (+ 2693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:13.604426 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:13.604893 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:13.605487 (+ 594us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:13.607387 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:13.608871 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:13.610845 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:13.611643 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:13.611822 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:13.616902 (+ 5080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:13.616912 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:13.616930 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:13.616979 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:13.620571 (+ 3592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:13.621543 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:13.621552 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:13.621560 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:13.621755 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:13.622197 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:13.622201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:13.622203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:13.622792 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:13.623142 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:13.623607 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 01:44:13.624087 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":452,"threads_started":1}
I20260811 01:44:13.632397 24413 raft_consensus.cc:348] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "926939ec8ecc414db40558db089b1268" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40057 } }
I20260811 01:44:13.632871 24413 raft_consensus.cc:374] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:13.632993 24413 raft_consensus.cc:729] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926939ec8ecc414db40558db089b1268, State: Initialized, Role: FOLLOWER
I20260811 01:44:13.633678 24413 consensus_queue.cc:260] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "926939ec8ecc414db40558db089b1268" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40057 } }
I20260811 01:44:13.633962 24413 raft_consensus.cc:388] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:44:13.634123 24413 raft_consensus.cc:482] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:44:13.634318 24413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:13.636826 24413 raft_consensus.cc:504] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "926939ec8ecc414db40558db089b1268" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40057 } }
I20260811 01:44:13.637527 24413 leader_election.cc:302] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 926939ec8ecc414db40558db089b1268; no voters:
I20260811 01:44:13.638403 24413 leader_election.cc:288] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:44:13.638556 24419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:13.639580 24419 raft_consensus.cc:686] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268 [term 1 LEADER]: Becoming Leader. State: Replica: 926939ec8ecc414db40558db089b1268, State: Running, Role: LEADER
I20260811 01:44:13.640204 24419 consensus_queue.cc:237] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "926939ec8ecc414db40558db089b1268" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40057 } }
I20260811 01:44:13.640784 24413 sys_catalog.cc:564] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:44:13.643743 24421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 926939ec8ecc414db40558db089b1268. Latest consensus state: current_term: 1 leader_uuid: "926939ec8ecc414db40558db089b1268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "926939ec8ecc414db40558db089b1268" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40057 } } }
I20260811 01:44:13.643844 24420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "926939ec8ecc414db40558db089b1268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "926939ec8ecc414db40558db089b1268" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40057 } } }
I20260811 01:44:13.644505 24420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268 [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:13.645007 24421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268 [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:13.648389 24428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:44:13.654273 24428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:44:13.664949 24428 catalog_manager.cc:1347] Generated new cluster ID: 4415c85eafc1495f9a5da83ee0a7fb71
I20260811 01:44:13.665103 24428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:44:13.681545 24428 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:44:13.683164 24428 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:44:13.697656 24428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268: Generated new TSK 0
I20260811 01:44:13.698863 24428 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
I20260811 01:44:13.773233 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40057
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:14.124940 24440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:14.125330 24440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:14.125442 24440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:14.132884 24440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:14.133014 24440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:14.133107 24440 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 01:44:14.133224 24440 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 01:44:14.133366 24440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:44:14.143932 24440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.214.126:40057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:14.145541 24440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:14.147014 24440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:14.161679 24448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:14.161769 24445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:14.161742 24446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:14.162636 24440 server_base.cc:1034] running on GCE node
I20260811 01:44:14.163267 24440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:14.164228 24440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:14.165494 24440 hybrid_clock.cc:648] HybridClock initialized: now 1786412654165448 us; error 34 us; skew 500 ppm
I20260811 01:44:14.165896 24440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:14 dist-test-slave-9gcw krb5kdc[24307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412654, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:14.169296 24440 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:44:14.170810 24440 webserver.cc:460] Webserver started at http://127.21.214.65:34863/ using document root <none> and password file <none>
I20260811 01:44:14.171509 24440 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:14.171613 24440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:14.171907 24440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:14.175089 24440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "0d35f905fd45404998f016ad8d1a1460"
format_stamp: "Formatted at 2026-08-11 01:44:14 on dist-test-slave-9gcw"
server_key: "5dd12595e2a82c3b4c5ad828abc006a7"
server_key_iv: "e46f28af1f9c9f4a94e3766d71ead2b1"
server_key_version: "encryptionkey@0"
I20260811 01:44:14.175858 24440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "0d35f905fd45404998f016ad8d1a1460"
format_stamp: "Formatted at 2026-08-11 01:44:14 on dist-test-slave-9gcw"
server_key: "5dd12595e2a82c3b4c5ad828abc006a7"
server_key_iv: "e46f28af1f9c9f4a94e3766d71ead2b1"
server_key_version: "encryptionkey@0"
I20260811 01:44:14.181843 24440 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 01:44:14.185469 24456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:14.186905 24440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 01:44:14.187098 24440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "0d35f905fd45404998f016ad8d1a1460"
format_stamp: "Formatted at 2026-08-11 01:44:14 on dist-test-slave-9gcw"
server_key: "5dd12595e2a82c3b4c5ad828abc006a7"
server_key_iv: "e46f28af1f9c9f4a94e3766d71ead2b1"
server_key_version: "encryptionkey@0"
I20260811 01:44:14.187325 24440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:14.204766 24440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:14.209026 24440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:14.209384 24440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:14.212134 24440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:14.212306 24440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:14.212447 24440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:14.212543 24440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 01:44:14 dist-test-slave-9gcw krb5kdc[24307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412654, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:14.237843 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:14.212983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:36823 (local address 127.21.214.126:40057)
0811 01:44:14.213576 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:14.213584 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:14.214870 (+ 1286us) server_negotiation.cc:407] Connection header received
0811 01:44:14.217110 (+ 2240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:14.217116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:14.217235 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:14.217370 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:14.219627 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:14.220719 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:14.223510 (+ 2791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:14.223936 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:14.224084 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:14.229697 (+ 5613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:14.229706 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:14.229711 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:14.229753 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:14.233431 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:14.234621 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:14.234629 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:14.234634 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:14.234800 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:14.235492 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:14.235497 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:14.235501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:14.235841 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:14.236015 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:14.237221 (+ 1206us) server_negotiation.cc:299] Negotiation successful
0811 01:44:14.237464 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":313,"threads_started":1}
I20260811 01:44:14.238991 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:14.213695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40057 (local address 127.21.214.65:36823)
0811 01:44:14.214665 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:14.214717 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:14.216751 (+ 2034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:14.217631 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:14.217640 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:14.218427 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:14.219404 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:14.219441 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:14.220911 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:14.220922 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:14.223348 (+ 2426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:14.223365 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:14.224188 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:14.224198 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:14.224553 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:14.225479 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:14.225506 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:14.229377 (+ 3871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:14.233656 (+ 4279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:14.233666 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:14.233689 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:14.234427 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:14.234931 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:14.234936 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:14.234940 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:14.235325 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:14.236107 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:14.236120 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:14.236735 (+ 615us) client_negotiation.cc:769] Sending connection context
0811 01:44:14.237078 (+ 343us) client_negotiation.cc:240] Negotiation successful
0811 01:44:14.237726 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":413,"threads_started":1}
I20260811 01:44:14.262370 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:14.246929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:45761 (local address 127.21.214.126:40057)
0811 01:44:14.247129 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:14.247136 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:14.247196 (+ 60us) server_negotiation.cc:407] Connection header received
0811 01:44:14.247444 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:14.247452 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:14.247556 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:14.247663 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:14.248765 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:14.249688 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:14.251887 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:14.252478 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:14.252630 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:14.255134 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:14.255138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:14.255142 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:14.255174 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:14.258787 (+ 3613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:14.259937 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:14.259947 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:14.259953 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:14.260147 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:14.260654 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:14.260661 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:14.260668 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:14.261063 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:14.261302 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:14.261909 (+ 607us) server_negotiation.cc:299] Negotiation successful
0811 01:44:14.262099 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 01:44:14.262733 24556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:14.246456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40057 (local address 127.21.214.65:45761)
0811 01:44:14.247109 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:14.247132 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:14.247282 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:14.247688 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:14.247693 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:14.248261 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:14.248616 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:14.248629 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:14.249878 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:14.249885 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:14.251699 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:14.251727 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:14.252678 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:14.252685 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:14.252882 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:14.253582 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:14.253602 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:14.254928 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:14.258950 (+ 4022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:14.258961 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:14.258968 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:14.259772 (+ 804us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:14.260315 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:14.260322 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:14.260325 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:14.260455 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:14.261239 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:14.261252 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:14.261607 (+ 355us) client_negotiation.cc:769] Sending connection context
0811 01:44:14.261803 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 01:44:14.262153 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":536,"threads_started":1}
I20260811 01:44:14.269960 24440 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:44147
I20260811 01:44:14.270002 24592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:44147 every 8 connection(s)
I20260811 01:44:14.271281 24440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:44:14.273516 24593 heartbeater.cc:347] Connected to a master server at 127.21.214.126:40057
I20260811 01:44:14.273763 24593 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:14.274469 24593 heartbeater.cc:511] Master 127.21.214.126:40057 requested a full tablet report, sending...
I20260811 01:44:14.276682 24359 ts_manager.cc:194] Registered new tserver with Master: 0d35f905fd45404998f016ad8d1a1460 (127.21.214.65:44147)
I20260811 01:44:14.276747 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 24440
I20260811 01:44:14.276965 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:44:14.277817 22361 external_mini_cluster.cc:1428] Setting key 77fb0fbfc88206116670f20281ea2c8d
I20260811 01:44:14.278867 24359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:36823
WARNING: no policy specified for kudu/127.21.214.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
I20260811 01:44:14.329658 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40057
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:14.675043 24597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:14.675491 24597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:14.675686 24597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:14.684175 24597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:14.684351 24597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:14.684521 24597 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 01:44:14.684620 24597 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 01:44:14.684754 24597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:44:14.695981 24597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.214.126:40057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:14.697527 24597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:14.698958 24597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:14.714119 24603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:14.714035 24605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:14.714082 24602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:14.715024 24597 server_base.cc:1034] running on GCE node
I20260811 01:44:14.715725 24597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:14.716825 24597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:14.718067 24597 hybrid_clock.cc:648] HybridClock initialized: now 1786412654717978 us; error 85 us; skew 500 ppm
I20260811 01:44:14.718441 24597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:14 dist-test-slave-9gcw krb5kdc[24307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412654, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:14.722576 24597 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:44:14.724239 24597 webserver.cc:460] Webserver started at http://127.21.214.66:39507/ using document root <none> and password file <none>
I20260811 01:44:14.724970 24597 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:14.725087 24597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:14.725490 24597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:14.728783 24597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "d113db0ec3fb4739837c7c9bd832c37c"
format_stamp: "Formatted at 2026-08-11 01:44:14 on dist-test-slave-9gcw"
server_key: "aad2676efcd35dbcd65e9fc85cf4f72a"
server_key_iv: "aa30dda3a502ed8cc24b7e68f7ad83c9"
server_key_version: "encryptionkey@0"
I20260811 01:44:14.729692 24597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "d113db0ec3fb4739837c7c9bd832c37c"
format_stamp: "Formatted at 2026-08-11 01:44:14 on dist-test-slave-9gcw"
server_key: "aad2676efcd35dbcd65e9fc85cf4f72a"
server_key_iv: "aa30dda3a502ed8cc24b7e68f7ad83c9"
server_key_version: "encryptionkey@0"
I20260811 01:44:14.735733 24597 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 01:44:14.740007 24615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:14.741627 24597 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 01:44:14.741880 24597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "d113db0ec3fb4739837c7c9bd832c37c"
format_stamp: "Formatted at 2026-08-11 01:44:14 on dist-test-slave-9gcw"
server_key: "aad2676efcd35dbcd65e9fc85cf4f72a"
server_key_iv: "aa30dda3a502ed8cc24b7e68f7ad83c9"
server_key_version: "encryptionkey@0"
I20260811 01:44:14.742105 24597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:14.784549 24597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:14.789724 24597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:14.790066 24597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:14.793248 24597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:14.793380 24597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:14.793498 24597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:14.793592 24597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 01:44:14 dist-test-slave-9gcw krb5kdc[24307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412654, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:14.819643 24624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:14.793956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:58285 (local address 127.21.214.126:40057)
0811 01:44:14.794446 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:14.794451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:14.796396 (+ 1945us) server_negotiation.cc:407] Connection header received
0811 01:44:14.798619 (+ 2223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:14.798624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:14.798717 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:14.798866 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:14.801538 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:14.802712 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:14.805691 (+ 2979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:14.806136 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:14.806294 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:14.811554 (+ 5260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:14.811559 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:14.811562 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:14.811599 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:14.815105 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:14.816279 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:14.816288 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:14.816294 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:14.816480 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:14.817190 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:14.817199 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:14.817205 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:14.817697 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:14.817927 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:14.818695 (+ 768us) server_negotiation.cc:299] Negotiation successful
0811 01:44:14.819036 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":265,"threads_started":1}
I20260811 01:44:14.820264 24625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:14.795355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40057 (local address 127.21.214.66:58285)
0811 01:44:14.796206 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:14.796253 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:14.798302 (+ 2049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:14.799344 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:14.799356 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:14.800242 (+ 886us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:14.801277 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:14.801309 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:14.802882 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:14.802890 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:14.805511 (+ 2621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:14.805528 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:14.806402 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:14.806410 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:14.806635 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:14.807499 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:14.807541 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:14.811283 (+ 3742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:14.815302 (+ 4019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:14.815309 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:14.815325 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:14.816063 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:14.816615 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:14.816624 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:14.816628 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:14.816970 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:14.817858 (+ 888us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:14.817865 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:14.818256 (+ 391us) client_negotiation.cc:769] Sending connection context
0811 01:44:14.818517 (+ 261us) client_negotiation.cc:240] Negotiation successful
0811 01:44:14.818954 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":373,"threads_started":1}
I20260811 01:44:14.844422 24703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:14.827332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40057 (local address 127.21.214.66:50485)
0811 01:44:14.828096 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:14.828123 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:14.828318 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:14.828693 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:14.828701 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:14.829508 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:14.829996 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:14.830013 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:14.831078 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:14.831087 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:14.833379 (+ 2292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:14.833394 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:14.834313 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:14.834321 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:14.834604 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:14.835543 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:14.835580 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:14.837138 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:14.840728 (+ 3590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:14.840737 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:14.840744 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:14.841609 (+ 865us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:14.842179 (+ 570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:14.842187 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:14.842193 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:14.842340 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:14.842950 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:14.842955 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:14.843170 (+ 215us) client_negotiation.cc:769] Sending connection context
0811 01:44:14.843345 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 01:44:14.843748 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":383,"threads_started":1}
I20260811 01:44:14.844880 24624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:14.827344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:50485 (local address 127.21.214.126:40057)
0811 01:44:14.827603 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:14.827613 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:14.828241 (+ 628us) server_negotiation.cc:407] Connection header received
0811 01:44:14.828449 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:14.828453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:14.828537 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:14.828622 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:14.830173 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:14.830916 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:14.833553 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:14.834071 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:14.834227 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:14.837411 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:14.837418 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:14.837424 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:14.837471 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:14.840585 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:14.841778 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:14.841783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:14.841786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:14.841956 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:14.842488 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:14.842491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:14.842493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:14.842820 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:14.842984 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:14.844263 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0811 01:44:14.844540 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 01:44:14.850222 24597 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:33981
I20260811 01:44:14.850282 24753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:33981 every 8 connection(s)
I20260811 01:44:14.851702 24597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:44:14.852706 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 24597
I20260811 01:44:14.852900 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:44:14.853493 22361 external_mini_cluster.cc:1428] Setting key 80f84d44d6f97796fc74b5e276dedd00
I20260811 01:44:14.853940 24754 heartbeater.cc:347] Connected to a master server at 127.21.214.126:40057
I20260811 01:44:14.854185 24754 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:14.854997 24754 heartbeater.cc:511] Master 127.21.214.126:40057 requested a full tablet report, sending...
I20260811 01:44:14.856503 24358 ts_manager.cc:194] Registered new tserver with Master: d113db0ec3fb4739837c7c9bd832c37c (127.21.214.66:33981)
I20260811 01:44:14.857515 24358 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:58285
WARNING: no policy specified for kudu/127.21.214.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
I20260811 01:44:14.908422 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40057
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:15.267189 24760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:15.267583 24760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:15.267791 24760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:15.275480 24760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:15.275653 24760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:15.275830 24760 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 01:44:15.275920 24760 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 01:44:15.276063 24760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:44:15.281908 24593 heartbeater.cc:503] Master 127.21.214.126:40057 was elected leader, sending a full tablet report...
I20260811 01:44:15.286749 24760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.214.126:40057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:15.288229 24760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:15.289700 24760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:15.304558 24767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:15.304616 24768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:15.304639 24770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:15.305769 24760 server_base.cc:1034] running on GCE node
I20260811 01:44:15.306480 24760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:15.307592 24760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:15.308914 24760 hybrid_clock.cc:648] HybridClock initialized: now 1786412655308841 us; error 73 us; skew 500 ppm
I20260811 01:44:15.309368 24760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:15 dist-test-slave-9gcw krb5kdc[24307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412655, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:15.312916 24760 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:44:15.314616 24760 webserver.cc:460] Webserver started at http://127.21.214.67:44325/ using document root <none> and password file <none>
I20260811 01:44:15.315382 24760 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:15.315541 24760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:15.315896 24760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:15.319116 24760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "dd1b1cec455448b0ad426757b43c3b93"
format_stamp: "Formatted at 2026-08-11 01:44:15 on dist-test-slave-9gcw"
server_key: "4311c0f0e388e317cdde0ff6cc22dd3b"
server_key_iv: "aab7774707cf132de317476b86251697"
server_key_version: "encryptionkey@0"
I20260811 01:44:15.319955 24760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "dd1b1cec455448b0ad426757b43c3b93"
format_stamp: "Formatted at 2026-08-11 01:44:15 on dist-test-slave-9gcw"
server_key: "4311c0f0e388e317cdde0ff6cc22dd3b"
server_key_iv: "aab7774707cf132de317476b86251697"
server_key_version: "encryptionkey@0"
I20260811 01:44:15.326238 24760 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 01:44:15.330072 24779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:15.331565 24760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:44:15.331800 24760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "dd1b1cec455448b0ad426757b43c3b93"
format_stamp: "Formatted at 2026-08-11 01:44:15 on dist-test-slave-9gcw"
server_key: "4311c0f0e388e317cdde0ff6cc22dd3b"
server_key_iv: "aab7774707cf132de317476b86251697"
server_key_version: "encryptionkey@0"
I20260811 01:44:15.332038 24760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:15.354014 24760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:15.359380 24760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:15.359754 24760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:15.363003 24760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:15.363154 24760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:15.363332 24760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:15.363435 24760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 01:44:15 dist-test-slave-9gcw krb5kdc[24307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412655, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:15.387048 24788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.363435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:59735 (local address 127.21.214.126:40057)
0811 01:44:15.363933 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:15.363940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:15.365226 (+ 1286us) server_negotiation.cc:407] Connection header received
0811 01:44:15.367521 (+ 2295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:15.367531 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:15.367662 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:15.367822 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:15.370373 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.371518 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.374242 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.374893 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.375046 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.380325 (+ 5279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:15.380330 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:15.380334 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:15.380393 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:15.383552 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:15.384392 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:15.384399 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:15.384408 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:15.384541 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:15.385033 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:15.385037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:15.385039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:15.385397 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:15.385559 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:15.386360 (+ 801us) server_negotiation.cc:299] Negotiation successful
0811 01:44:15.386619 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":251,"threads_started":1}
I20260811 01:44:15.388320 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.364059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40057 (local address 127.21.214.67:59735)
0811 01:44:15.365006 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:15.365056 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:15.367073 (+ 2017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:15.368150 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:15.368169 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:15.369015 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:15.370159 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.370183 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.371704 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.371708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.374054 (+ 2346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.374068 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.375230 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.375245 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.375471 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.376303 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:15.376355 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:15.379984 (+ 3629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:15.383732 (+ 3748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:15.383743 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:15.383765 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:15.384246 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:15.384666 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:15.384672 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:15.384676 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:15.384895 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:15.385539 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:15.385547 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:15.385955 (+ 408us) client_negotiation.cc:769] Sending connection context
0811 01:44:15.386252 (+ 297us) client_negotiation.cc:240] Negotiation successful
0811 01:44:15.386853 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":389,"threads_started":1}
I20260811 01:44:15.415474 24854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.395861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40057 (local address 127.21.214.67:33165)
0811 01:44:15.397735 (+ 1874us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:15.397751 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:15.397883 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:15.398354 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:15.398360 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:15.399476 (+ 1116us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:15.400008 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.400027 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.401826 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.401834 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.403150 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.403170 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.404552 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.404561 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.404814 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.405683 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:15.405717 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:15.407286 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:15.412040 (+ 4754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:15.412049 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:15.412055 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:15.412723 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:15.413286 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:15.413294 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:15.413303 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:15.413519 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:15.414360 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:15.414367 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:15.414612 (+ 245us) client_negotiation.cc:769] Sending connection context
0811 01:44:15.414752 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 01:44:15.415027 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1709,"thread_start_us":1537,"threads_started":1}
I20260811 01:44:15.415768 24788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.395896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:33165 (local address 127.21.214.126:40057)
0811 01:44:15.396157 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:15.396167 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:15.397857 (+ 1690us) server_negotiation.cc:407] Connection header received
0811 01:44:15.398029 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:15.398038 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:15.398162 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:15.398321 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:15.400240 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.401584 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.403386 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.404233 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.404450 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.407571 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:15.407713 (+ 142us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:15.407723 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:15.407781 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:15.411882 (+ 4101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:15.412872 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:15.412881 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:15.412887 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:15.413069 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:15.413673 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:15.413678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:15.413681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:15.414122 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:15.414352 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:15.414915 (+ 563us) server_negotiation.cc:299] Negotiation successful
0811 01:44:15.415293 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 01:44:15.422406 24760 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:45639
I20260811 01:44:15.422492 24920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:45639 every 8 connection(s)
I20260811 01:44:15.423835 24760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:44:15.426411 24921 heartbeater.cc:347] Connected to a master server at 127.21.214.126:40057
I20260811 01:44:15.426611 24921 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:15.427307 24921 heartbeater.cc:511] Master 127.21.214.126:40057 requested a full tablet report, sending...
I20260811 01:44:15.428629 24358 ts_manager.cc:194] Registered new tserver with Master: dd1b1cec455448b0ad426757b43c3b93 (127.21.214.67:45639)
I20260811 01:44:15.429548 24358 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:59735
I20260811 01:44:15.432928 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 24760
I20260811 01:44:15.433125 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:44:15.433681 22361 external_mini_cluster.cc:1428] Setting key 693beadac9a2c93de7f425dce608f711
I20260811 01:44:15.436931 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 01:44:15 dist-test-slave-9gcw krb5kdc[24307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 01:44:15 dist-test-slave-9gcw krb5kdc[24307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:15.493095 24788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.478233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49476 (local address 127.21.214.126:40057)
0811 01:44:15.478445 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:15.478450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:15.478903 (+ 453us) server_negotiation.cc:407] Connection header received
0811 01:44:15.479076 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:15.479080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:15.479147 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:15.479243 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:15.480701 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.481521 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.482767 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.483181 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.483302 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.487374 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:15.487379 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:15.487382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:15.487419 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:15.490209 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:15.491102 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:15.491107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:15.491109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:15.491235 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:15.491661 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:15.491668 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:15.491673 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:15.492079 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:15.492305 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:15.492595 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 01:44:15.492771 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 01:44:15.497946 24358 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:44:15.506266 24358 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:44:15.549988 24938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.520403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53420 (local address 127.21.214.66:33981)
0811 01:44:15.538915 (+ 18512us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:15.538930 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:15.541962 (+ 3032us) server_negotiation.cc:407] Connection header received
0811 01:44:15.542168 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:15.542179 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:15.542479 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:15.542678 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:15.543619 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.544870 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.547558 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.548595 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.548797 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.549026 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:15.549462 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 01:44:15.549661 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18253,"thread_start_us":18558,"threads_started":1}
I20260811 01:44:15.550840 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.519751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:33981 (local address 127.0.0.1:53420)
0811 01:44:15.541792 (+ 22041us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:15.541821 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:15.541989 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:15.542922 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:15.542926 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:15.542947 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:15.543396 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.543410 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.545050 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.545056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.547373 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.547386 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.548843 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.548849 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.549028 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.549097 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 01:44:15.550315 (+ 1218us) client_negotiation.cc:240] Negotiation successful
0811 01:44:15.550444 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21848,"thread_start_us":22515,"threads_started":1}
I20260811 01:44:15.551438 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.519587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53500 (local address 127.21.214.65:44147)
0811 01:44:15.535181 (+ 15594us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:15.535194 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:15.540438 (+ 5244us) server_negotiation.cc:407] Connection header received
0811 01:44:15.540571 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:15.540582 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:15.540927 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:15.542273 (+ 1346us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:15.542631 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.544433 (+ 1802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.547339 (+ 2906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.549049 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.549264 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.549479 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:15.550753 (+ 1274us) server_negotiation.cc:299] Negotiation successful
0811 01:44:15.551170 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15342,"thread_start_us":15168,"threads_started":1}
I20260811 01:44:15.551947 24934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.519359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:44147 (local address 127.0.0.1:53500)
0811 01:44:15.539965 (+ 20606us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:15.540017 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:15.540301 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:15.541446 (+ 1145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:15.541459 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:15.541511 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:15.542058 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.542075 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.544582 (+ 2507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.544589 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.547157 (+ 2568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.547170 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.549332 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.549340 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.549545 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.549604 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 01:44:15.549700 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 01:44:15.549923 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20305,"mutex_wait_us":61,"thread_start_us":20105,"threads_started":1}
I20260811 01:44:15.553061 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.519991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:45639 (local address 127.0.0.1:57708)
0811 01:44:15.542573 (+ 22582us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:15.542594 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:15.542730 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:15.543777 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:15.543782 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:15.543809 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:15.544160 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.544171 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.547391 (+ 3220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.547400 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.549799 (+ 2399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.549818 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.552201 (+ 2383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.552209 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.552423 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.552460 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 01:44:15.552551 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 01:44:15.552701 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22457,"thread_start_us":21458,"threads_started":1}
I20260811 01:44:15.553799 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.520778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57708 (local address 127.21.214.67:45639)
0811 01:44:15.541169 (+ 20391us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:15.541182 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:15.542877 (+ 1695us) server_negotiation.cc:407] Connection header received
0811 01:44:15.543021 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:15.543028 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:15.543283 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:15.543471 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:15.544435 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.546268 (+ 1833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.550104 (+ 3836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.551876 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.552042 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.553279 (+ 1237us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:15.553361 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 01:44:15.553550 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20146,"thread_start_us":19950,"threads_started":1}
I20260811 01:44:15.555409 24671 tablet_service.cc:1467] Processing CreateTablet for tablet dad3a3348f054d0eacf69e0fb8d28c81 (DEFAULT_TABLE table=test-table [id=253fab3c49c24070bfb7787516a9bcef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:15.555794 24516 tablet_service.cc:1467] Processing CreateTablet for tablet dad3a3348f054d0eacf69e0fb8d28c81 (DEFAULT_TABLE table=test-table [id=253fab3c49c24070bfb7787516a9bcef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:15.557066 24516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dad3a3348f054d0eacf69e0fb8d28c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:15.557055 24671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dad3a3348f054d0eacf69e0fb8d28c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:15.563988 24836 tablet_service.cc:1467] Processing CreateTablet for tablet dad3a3348f054d0eacf69e0fb8d28c81 (DEFAULT_TABLE table=test-table [id=253fab3c49c24070bfb7787516a9bcef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:15.565598 24836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dad3a3348f054d0eacf69e0fb8d28c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:15.570195 24940 tablet_bootstrap.cc:492] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460: Bootstrap starting.
I20260811 01:44:15.573469 24941 tablet_bootstrap.cc:492] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c: Bootstrap starting.
I20260811 01:44:15.575093 24940 tablet_bootstrap.cc:654] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:15.576608 24940 log.cc:824] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:15.577165 24941 tablet_bootstrap.cc:654] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:15.578375 24941 log.cc:824] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:15.580127 24940 tablet_bootstrap.cc:492] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460: No bootstrap required, opened a new log
I20260811 01:44:15.580426 24940 ts_tablet_manager.cc:1397] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260811 01:44:15.580881 24944 tablet_bootstrap.cc:492] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93: Bootstrap starting.
I20260811 01:44:15.581563 24941 tablet_bootstrap.cc:492] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c: No bootstrap required, opened a new log
I20260811 01:44:15.581845 24941 ts_tablet_manager.cc:1397] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260811 01:44:15.584208 24944 tablet_bootstrap.cc:654] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:15.585307 24944 log.cc:824] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:15.587951 24944 tablet_bootstrap.cc:492] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93: No bootstrap required, opened a new log
I20260811 01:44:15.588236 24944 ts_tablet_manager.cc:1397] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 01:44:15.592475 24940 raft_consensus.cc:348] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:15.592993 24940 raft_consensus.cc:374] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:15.592739 24941 raft_consensus.cc:348] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:15.593091 24940 raft_consensus.cc:729] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d35f905fd45404998f016ad8d1a1460, State: Initialized, Role: FOLLOWER
I20260811 01:44:15.593197 24941 raft_consensus.cc:374] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:15.593286 24941 raft_consensus.cc:729] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d113db0ec3fb4739837c7c9bd832c37c, State: Initialized, Role: FOLLOWER
I20260811 01:44:15.593978 24941 consensus_queue.cc:260] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:15.593973 24940 consensus_queue.cc:260] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:15.598094 24944 raft_consensus.cc:348] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:15.598500 24944 raft_consensus.cc:374] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:15.598585 24944 raft_consensus.cc:729] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd1b1cec455448b0ad426757b43c3b93, State: Initialized, Role: FOLLOWER
I20260811 01:44:15.599406 24944 consensus_queue.cc:260] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:15.601257 24754 heartbeater.cc:503] Master 127.21.214.126:40057 was elected leader, sending a full tablet report...
I20260811 01:44:15.602406 24941 ts_tablet_manager.cc:1428] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c: Time spent starting tablet: real 0.020s user 0.010s sys 0.011s
I20260811 01:44:15.603091 24940 ts_tablet_manager.cc:1428] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460: Time spent starting tablet: real 0.023s user 0.018s sys 0.005s
W20260811 01:44:15.603662 24757 tablet.cc:2369] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:44:15.608685 24921 heartbeater.cc:503] Master 127.21.214.126:40057 was elected leader, sending a full tablet report...
I20260811 01:44:15.609566 24944 ts_tablet_manager.cc:1428] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93: Time spent starting tablet: real 0.021s user 0.016s sys 0.004s
I20260811 01:44:15.610090 24949 raft_consensus.cc:482] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:44:15.610399 24949 raft_consensus.cc:504] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:15.619127 24949 leader_election.cc:288] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d113db0ec3fb4739837c7c9bd832c37c (127.21.214.66:33981), dd1b1cec455448b0ad426757b43c3b93 (127.21.214.67:45639)
I20260811 01:44:15.626309 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.613352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:60039 (local address 127.21.214.67:45639)
0811 01:44:15.613563 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:15.613571 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:15.620199 (+ 6628us) server_negotiation.cc:407] Connection header received
0811 01:44:15.620416 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:15.620424 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:15.620522 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:15.620624 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:15.621544 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.622453 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.624469 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.625327 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.625523 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.625608 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:15.625898 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 01:44:15.626029 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 01:44:15.626384 24951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.612774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:45639 (local address 127.21.214.65:60039)
0811 01:44:15.620005 (+ 7231us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:15.620024 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:15.620250 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:15.620823 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:15.620828 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:15.620855 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:15.621389 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.621400 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.622603 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.622608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.624284 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.624302 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.625556 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.625562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.625721 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.625746 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 01:44:15.625867 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 01:44:15.626007 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7083,"thread_start_us":6826,"threads_started":1}
I20260811 01:44:15.627059 24938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.613340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:41455 (local address 127.21.214.66:33981)
0811 01:44:15.613595 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:15.613605 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:15.621211 (+ 7606us) server_negotiation.cc:407] Connection header received
0811 01:44:15.621375 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:15.621380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:15.621458 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:15.621545 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:15.622296 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.623311 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.625055 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.625912 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.626114 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.626199 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:15.626687 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 01:44:15.626839 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 01:44:15.627274 24952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.612813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:33981 (local address 127.21.214.65:41455)
0811 01:44:15.621010 (+ 8197us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:15.621040 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:15.621222 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:15.621720 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:15.621725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:15.621752 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:15.622112 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.622123 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.623459 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.623464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.624889 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.624902 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.626155 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.626162 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.626311 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.626334 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 01:44:15.626632 (+ 298us) client_negotiation.cc:240] Negotiation successful
0811 01:44:15.626790 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8025,"thread_start_us":7708,"threads_started":1}
I20260811 01:44:15.627655 24866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad3a3348f054d0eacf69e0fb8d28c81" candidate_uuid: "0d35f905fd45404998f016ad8d1a1460" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd1b1cec455448b0ad426757b43c3b93" is_pre_election: true
I20260811 01:44:15.628324 24866 raft_consensus.cc:2454] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d35f905fd45404998f016ad8d1a1460 in term 0.
I20260811 01:44:15.628337 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad3a3348f054d0eacf69e0fb8d28c81" candidate_uuid: "0d35f905fd45404998f016ad8d1a1460" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d113db0ec3fb4739837c7c9bd832c37c" is_pre_election: true
I20260811 01:44:15.628891 24693 raft_consensus.cc:2454] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d35f905fd45404998f016ad8d1a1460 in term 0.
I20260811 01:44:15.629216 24457 leader_election.cc:302] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d35f905fd45404998f016ad8d1a1460, dd1b1cec455448b0ad426757b43c3b93; no voters:
I20260811 01:44:15.629733 24949 raft_consensus.cc:2781] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:44:15.629901 24949 raft_consensus.cc:482] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:44:15.630018 24949 raft_consensus.cc:3037] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:15.632925 24949 raft_consensus.cc:504] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:15.633733 24949 leader_election.cc:288] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [CANDIDATE]: Term 1 election: Requested vote from peers d113db0ec3fb4739837c7c9bd832c37c (127.21.214.66:33981), dd1b1cec455448b0ad426757b43c3b93 (127.21.214.67:45639)
I20260811 01:44:15.634198 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad3a3348f054d0eacf69e0fb8d28c81" candidate_uuid: "0d35f905fd45404998f016ad8d1a1460" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d113db0ec3fb4739837c7c9bd832c37c"
I20260811 01:44:15.634279 24866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad3a3348f054d0eacf69e0fb8d28c81" candidate_uuid: "0d35f905fd45404998f016ad8d1a1460" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd1b1cec455448b0ad426757b43c3b93"
I20260811 01:44:15.634428 24693 raft_consensus.cc:3037] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:15.634531 24866 raft_consensus.cc:3037] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:15.636934 24693 raft_consensus.cc:2454] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d35f905fd45404998f016ad8d1a1460 in term 1.
I20260811 01:44:15.637266 24866 raft_consensus.cc:2454] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d35f905fd45404998f016ad8d1a1460 in term 1.
I20260811 01:44:15.637477 24458 leader_election.cc:302] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d35f905fd45404998f016ad8d1a1460, d113db0ec3fb4739837c7c9bd832c37c; no voters:
I20260811 01:44:15.637844 24949 raft_consensus.cc:2781] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:15.638322 24949 raft_consensus.cc:686] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [term 1 LEADER]: Becoming Leader. State: Replica: 0d35f905fd45404998f016ad8d1a1460, State: Running, Role: LEADER
I20260811 01:44:15.638816 24949 consensus_queue.cc:237] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:15.644086 24359 catalog_manager.cc:5495] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d35f905fd45404998f016ad8d1a1460 (127.21.214.65). New cstate: current_term: 1 leader_uuid: "0d35f905fd45404998f016ad8d1a1460" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } health_report { overall_health: UNKNOWN } } }
W20260811 01:44:15.675565 24922 tablet.cc:2369] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 01:44:15 dist-test-slave-9gcw krb5kdc[24307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412653, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:15.694988 24960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.680004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40057 (local address 127.0.0.1:49484)
0811 01:44:15.680680 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:15.680698 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:15.680880 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:15.681250 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:15.681256 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:15.681992 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:15.682278 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.682289 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.683396 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.683404 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.684390 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.684402 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.685259 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.685266 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.685408 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.686129 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:15.686147 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:15.688604 (+ 2457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:15.692100 (+ 3496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:15.692113 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:15.692120 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:15.692785 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:15.693250 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:15.693254 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:15.693256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:15.693404 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:15.694100 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:15.694106 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:15.694263 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 01:44:15.694409 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 01:44:15.694619 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":384,"threads_started":1}
I20260811 01:44:15.695354 24788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.679874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49484 (local address 127.21.214.126:40057)
0811 01:44:15.680093 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:15.680098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:15.680796 (+ 698us) server_negotiation.cc:407] Connection header received
0811 01:44:15.681005 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:15.681015 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:15.681072 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:15.681197 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:15.682428 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.683233 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.684549 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.685013 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.685152 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.688843 (+ 3691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:15.688848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:15.688851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:15.688883 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:15.691916 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:15.692920 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:15.692924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:15.692927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:15.693067 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:15.693547 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:15.693552 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:15.693554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:15.693972 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:15.694134 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:15.694544 (+ 410us) server_negotiation.cc:299] Negotiation successful
0811 01:44:15.694850 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 01:44:15.701124 24359 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:49484:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 01:44:15.702690 24359 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:44:15.710775 24516 tablet_service.cc:1467] Processing CreateTablet for tablet 313c16e33c6d453fbcfb8f6d4b650d10 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e1c9b3726d464059a2e2c80b1905e992]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 01:44:15.710632 24671 tablet_service.cc:1467] Processing CreateTablet for tablet 313c16e33c6d453fbcfb8f6d4b650d10 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e1c9b3726d464059a2e2c80b1905e992]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 01:44:15.711766 24516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 313c16e33c6d453fbcfb8f6d4b650d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:15.711777 24671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 313c16e33c6d453fbcfb8f6d4b650d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:15.711335 24836 tablet_service.cc:1467] Processing CreateTablet for tablet 313c16e33c6d453fbcfb8f6d4b650d10 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e1c9b3726d464059a2e2c80b1905e992]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 01:44:15.712189 24836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 313c16e33c6d453fbcfb8f6d4b650d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:15.717864 24941 tablet_bootstrap.cc:492] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c: Bootstrap starting.
I20260811 01:44:15.719616 24940 tablet_bootstrap.cc:492] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460: Bootstrap starting.
I20260811 01:44:15.719750 24941 tablet_bootstrap.cc:654] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:15.720546 24944 tablet_bootstrap.cc:492] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93: Bootstrap starting.
I20260811 01:44:15.721684 24940 tablet_bootstrap.cc:654] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:15.722318 24944 tablet_bootstrap.cc:654] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:15.723316 24941 tablet_bootstrap.cc:492] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c: No bootstrap required, opened a new log
I20260811 01:44:15.723490 24941 ts_tablet_manager.cc:1397] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20260811 01:44:15.724370 24941 raft_consensus.cc:348] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:15.724581 24940 tablet_bootstrap.cc:492] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460: No bootstrap required, opened a new log
I20260811 01:44:15.724650 24941 raft_consensus.cc:374] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:15.724725 24940 ts_tablet_manager.cc:1397] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 01:44:15.724756 24941 raft_consensus.cc:729] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d113db0ec3fb4739837c7c9bd832c37c, State: Initialized, Role: FOLLOWER
I20260811 01:44:15.725031 24941 consensus_queue.cc:260] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:15.725605 24940 raft_consensus.cc:348] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:15.725564 24947 tablet_replica.cc:439] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } } }
I20260811 01:44:15.725855 24940 raft_consensus.cc:374] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:15.725945 24940 raft_consensus.cc:729] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d35f905fd45404998f016ad8d1a1460, State: Initialized, Role: FOLLOWER
I20260811 01:44:15.725978 24947 tablet_replica.cc:442] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:44:15.726217 24940 consensus_queue.cc:260] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:15.726504 24941 ts_tablet_manager.cc:1428] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 01:44:15.726686 24949 tablet_replica.cc:439] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } } }
I20260811 01:44:15.726961 24949 tablet_replica.cc:442] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:44:15.728003 24940 ts_tablet_manager.cc:1428] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 01:44:15.730970 24944 tablet_bootstrap.cc:492] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93: No bootstrap required, opened a new log
I20260811 01:44:15.731194 24944 ts_tablet_manager.cc:1397] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260811 01:44:15.732038 24944 raft_consensus.cc:348] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:15.732261 24944 raft_consensus.cc:374] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:15.732383 24944 raft_consensus.cc:729] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd1b1cec455448b0ad426757b43c3b93, State: Initialized, Role: FOLLOWER
I20260811 01:44:15.732673 24944 consensus_queue.cc:260] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:15.733433 24944 ts_tablet_manager.cc:1428] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 01:44:15.733263 24950 tablet_replica.cc:439] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } } }
I20260811 01:44:15.733741 24950 tablet_replica.cc:442] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93: This TxnStatusTablet replica's current role is: FOLLOWER
W20260811 01:44:15.774135 24594 tablet.cc:2369] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:44:15.838356 24950 raft_consensus.cc:482] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:44:15.838601 24950 raft_consensus.cc:504] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:15.840360 24950 leader_election.cc:288] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d35f905fd45404998f016ad8d1a1460 (127.21.214.65:44147), d113db0ec3fb4739837c7c9bd832c37c (127.21.214.66:33981)
I20260811 01:44:15.847599 24938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.840850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:36109 (local address 127.21.214.66:33981)
0811 01:44:15.841061 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:15.841071 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:15.841101 (+ 30us) server_negotiation.cc:407] Connection header received
0811 01:44:15.841434 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:15.841439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:15.841508 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:15.841649 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:15.842242 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.843095 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.845103 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.846559 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.846697 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.846789 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:15.847230 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 01:44:15.847353 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 01:44:15.847676 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.840667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:33981 (local address 127.21.214.67:36109)
0811 01:44:15.840904 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:15.840927 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:15.841130 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:15.841706 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:15.841713 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:15.841742 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:15.841948 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.841958 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.843268 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.843274 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.844910 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.844930 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.846753 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.846761 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.847001 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.847041 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 01:44:15.847211 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 01:44:15.847367 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 01:44:15.848378 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "313c16e33c6d453fbcfb8f6d4b650d10" candidate_uuid: "dd1b1cec455448b0ad426757b43c3b93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d113db0ec3fb4739837c7c9bd832c37c" is_pre_election: true
I20260811 01:44:15.848690 24693 raft_consensus.cc:2454] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1b1cec455448b0ad426757b43c3b93 in term 0.
I20260811 01:44:15.849050 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.840923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:59987 (local address 127.21.214.65:44147)
0811 01:44:15.841275 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:15.841283 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:15.841814 (+ 531us) server_negotiation.cc:407] Connection header received
0811 01:44:15.842087 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:15.842116 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:15.842241 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:15.842381 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:15.842980 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.843756 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.846360 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.847879 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.848070 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.848177 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:15.848627 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 01:44:15.848784 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260811 01:44:15.849303 24964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.840667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:44147 (local address 127.21.214.67:59987)
0811 01:44:15.841680 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:15.841701 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:15.841900 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:15.842397 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:15.842402 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:15.842427 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:15.842806 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.842822 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.843908 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.843914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.846171 (+ 2257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.846189 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.848181 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.848191 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.848442 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.848471 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 01:44:15.848595 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 01:44:15.848739 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"mutex_wait_us":252,"thread_start_us":396,"threads_started":1}
I20260811 01:44:15.849334 24781 leader_election.cc:302] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d113db0ec3fb4739837c7c9bd832c37c, dd1b1cec455448b0ad426757b43c3b93; no voters:
I20260811 01:44:15.849946 24950 raft_consensus.cc:2781] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:44:15.850097 24950 raft_consensus.cc:482] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:44:15.850214 24950 raft_consensus.cc:3037] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:15.850632 24537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "313c16e33c6d453fbcfb8f6d4b650d10" candidate_uuid: "dd1b1cec455448b0ad426757b43c3b93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d35f905fd45404998f016ad8d1a1460" is_pre_election: true
I20260811 01:44:15.851186 24537 raft_consensus.cc:2454] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1b1cec455448b0ad426757b43c3b93 in term 0.
I20260811 01:44:15.853319 24950 raft_consensus.cc:504] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:15.854008 24950 leader_election.cc:288] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [CANDIDATE]: Term 1 election: Requested vote from peers 0d35f905fd45404998f016ad8d1a1460 (127.21.214.65:44147), d113db0ec3fb4739837c7c9bd832c37c (127.21.214.66:33981)
I20260811 01:44:15.854499 24537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "313c16e33c6d453fbcfb8f6d4b650d10" candidate_uuid: "dd1b1cec455448b0ad426757b43c3b93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d35f905fd45404998f016ad8d1a1460"
I20260811 01:44:15.854723 24537 raft_consensus.cc:3037] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:15.854759 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "313c16e33c6d453fbcfb8f6d4b650d10" candidate_uuid: "dd1b1cec455448b0ad426757b43c3b93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d113db0ec3fb4739837c7c9bd832c37c"
I20260811 01:44:15.855067 24693 raft_consensus.cc:3037] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:15.857779 24693 raft_consensus.cc:2454] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1b1cec455448b0ad426757b43c3b93 in term 1.
I20260811 01:44:15.858249 24781 leader_election.cc:302] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d113db0ec3fb4739837c7c9bd832c37c, dd1b1cec455448b0ad426757b43c3b93; no voters:
I20260811 01:44:15.858268 24537 raft_consensus.cc:2454] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1b1cec455448b0ad426757b43c3b93 in term 1.
I20260811 01:44:15.858789 24950 raft_consensus.cc:2781] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:15.859354 24950 raft_consensus.cc:686] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [term 1 LEADER]: Becoming Leader. State: Replica: dd1b1cec455448b0ad426757b43c3b93, State: Running, Role: LEADER
I20260811 01:44:15.859936 24950 consensus_queue.cc:237] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:15.864141 24965 tablet_replica.cc:439] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93: TxnStatusTablet state changed. Reason: New leader dd1b1cec455448b0ad426757b43c3b93. Latest consensus state: current_term: 1 leader_uuid: "dd1b1cec455448b0ad426757b43c3b93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } } }
I20260811 01:44:15.864603 24965 tablet_replica.cc:442] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93: This TxnStatusTablet replica's current role is: LEADER
I20260811 01:44:15.865501 24967 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 01:44:15.865795 24967 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 01:44:15.866240 24359 catalog_manager.cc:5495] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd1b1cec455448b0ad426757b43c3b93 (127.21.214.67). New cstate: current_term: 1 leader_uuid: "dd1b1cec455448b0ad426757b43c3b93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } health_report { overall_health: HEALTHY } } }
I20260811 01:44:15.892020 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.884197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57724 (local address 127.21.214.67:45639)
0811 01:44:15.884594 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:15.884600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:15.884619 (+ 19us) server_negotiation.cc:407] Connection header received
0811 01:44:15.884996 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:15.885000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:15.885086 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:15.885330 (+ 244us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:44:15.885825 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.886797 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.888978 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.889523 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.889703 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.890354 (+ 651us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:44:15.891191 (+ 837us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:44:15.891362 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:15.891537 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 01:44:15.891636 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280}
I20260811 01:44:15.892158 24960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.884197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:45639 (local address 127.0.0.1:57724)
0811 01:44:15.884493 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:15.884521 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:15.884718 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:15.885396 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:15.885402 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:15.885419 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 01:44:15.885662 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.885676 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.886949 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.886956 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.888808 (+ 1852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.888826 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.889711 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.889719 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.890027 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.890126 (+ 99us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 01:44:15.891338 (+ 1212us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 01:44:15.891368 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 01:44:15.891474 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 01:44:15.891736 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 01:44:15.902244 24693 raft_consensus.cc:1261] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c [term 1 FOLLOWER]: Refusing update from remote peer dd1b1cec455448b0ad426757b43c3b93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:15.902222 24537 raft_consensus.cc:1261] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460 [term 1 FOLLOWER]: Refusing update from remote peer dd1b1cec455448b0ad426757b43c3b93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:15.903417 24950 consensus_queue.cc:1035] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:15.903806 24965 consensus_queue.cc:1035] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:15.916214 24949 tablet_replica.cc:439] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460: TxnStatusTablet state changed. Reason: New leader dd1b1cec455448b0ad426757b43c3b93. Latest consensus state: current_term: 1 leader_uuid: "dd1b1cec455448b0ad426757b43c3b93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } } }
I20260811 01:44:15.916606 24949 tablet_replica.cc:442] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:44:15.917425 24947 tablet_replica.cc:439] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c: TxnStatusTablet state changed. Reason: New leader dd1b1cec455448b0ad426757b43c3b93. Latest consensus state: current_term: 1 leader_uuid: "dd1b1cec455448b0ad426757b43c3b93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } } }
I20260811 01:44:15.917761 24947 tablet_replica.cc:442] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:44:15.921718 24970 mvcc.cc:205] Tried to move back new op lower bound from 7317146238563737600 to 7317146238406819840. Current Snapshot: MvccSnapshot[applied={T|T < 7317146238563737600}]
I20260811 01:44:15.923913 24965 tablet_replica.cc:439] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd1b1cec455448b0ad426757b43c3b93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } } }
I20260811 01:44:15.924229 24965 tablet_replica.cc:442] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93: This TxnStatusTablet replica's current role is: LEADER
I20260811 01:44:15.926896 24949 tablet_replica.cc:439] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd1b1cec455448b0ad426757b43c3b93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } } }
I20260811 01:44:15.927232 24949 tablet_replica.cc:442] T 313c16e33c6d453fbcfb8f6d4b650d10 P 0d35f905fd45404998f016ad8d1a1460: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:44:15.929435 24976 tablet_replica.cc:439] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd1b1cec455448b0ad426757b43c3b93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } } }
I20260811 01:44:15.930450 24976 tablet_replica.cc:442] T 313c16e33c6d453fbcfb8f6d4b650d10 P dd1b1cec455448b0ad426757b43c3b93: This TxnStatusTablet replica's current role is: LEADER
I20260811 01:44:15.933871 24947 tablet_replica.cc:439] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd1b1cec455448b0ad426757b43c3b93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } } }
I20260811 01:44:15.934170 24947 tablet_replica.cc:442] T 313c16e33c6d453fbcfb8f6d4b650d10 P d113db0ec3fb4739837c7c9bd832c37c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:44:15.942484 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.935042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53516 (local address 127.21.214.65:44147)
0811 01:44:15.935761 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:15.935768 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:15.935788 (+ 20us) server_negotiation.cc:407] Connection header received
0811 01:44:15.935882 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:15.935887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:15.935985 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:15.936157 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:44:15.936616 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.937880 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.939851 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.940296 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.940400 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.940965 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:44:15.941772 (+ 807us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:44:15.941948 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:15.942034 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 01:44:15.942107 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558}
I20260811 01:44:15.962813 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.955110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:47971 (local address 127.21.214.67:45639)
0811 01:44:15.955322 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:15.955327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:15.957634 (+ 2307us) server_negotiation.cc:407] Connection header received
0811 01:44:15.957889 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:15.957893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:15.957968 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:15.958101 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:44:15.958666 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.959426 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.960857 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.961371 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.961523 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.962006 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:44:15.962125 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:44:15.962237 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:15.962391 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 01:44:15.962476 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 01:44:15.962890 24984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:15.954968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:45639 (local address 127.21.214.65:47971)
0811 01:44:15.957514 (+ 2546us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:15.957532 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:15.957714 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:15.958146 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:15.958150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:15.958165 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 01:44:15.958503 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.958513 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.959565 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.959571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.960696 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:15.960708 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:15.961588 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:15.961595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:15.961725 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:15.961783 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 01:44:15.962238 (+ 455us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 01:44:15.962252 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 01:44:15.962364 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 01:44:15.962502 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2380,"thread_start_us":2120,"threads_started":1}
I20260811 01:44:15.973240 24693 raft_consensus.cc:1261] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c [term 1 FOLLOWER]: Refusing update from remote peer 0d35f905fd45404998f016ad8d1a1460: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:15.973510 24866 raft_consensus.cc:1261] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93 [term 1 FOLLOWER]: Refusing update from remote peer 0d35f905fd45404998f016ad8d1a1460: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:15.973958 24953 consensus_queue.cc:1035] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:15.974432 24949 consensus_queue.cc:1035] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:15.980744 24972 mvcc.cc:205] Tried to move back new op lower bound from 7317146238859849728 to 7317146237500358656. Current Snapshot: MvccSnapshot[applied={T|T < 7317146238859849728}]
I20260811 01:44:16.012964 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:16.006293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:37923 (local address 127.21.214.65:44147)
0811 01:44:16.006490 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:16.006496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:16.006699 (+ 203us) server_negotiation.cc:407] Connection header received
0811 01:44:16.006899 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:16.006905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:16.006994 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:16.007107 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:44:16.007645 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:16.008816 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:16.010899 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:16.011571 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:16.011720 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:16.012177 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:44:16.012317 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:44:16.012407 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:16.012565 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 01:44:16.012665 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 01:44:16.013299 24989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:16.005787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:44147 (local address 127.21.214.67:37923)
0811 01:44:16.006574 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:16.006600 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:16.006754 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:16.007135 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:16.007138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:16.007148 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 01:44:16.007472 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:16.007480 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:16.008975 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:16.008981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:16.010750 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:16.010763 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:16.011755 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:16.011759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:16.011890 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:16.011948 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 01:44:16.012416 (+ 468us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 01:44:16.012429 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 01:44:16.012520 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 01:44:16.012669 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":712,"threads_started":1}
I20260811 01:44:16.059736 24358 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:44:16.060071 24358 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:44:16.064704 24358 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268: Sending DeleteTablet for 3 replicas of tablet dad3a3348f054d0eacf69e0fb8d28c81
I20260811 01:44:16.066198 24516 tablet_service.cc:1514] Processing DeleteTablet for tablet dad3a3348f054d0eacf69e0fb8d28c81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:16 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:53500
I20260811 01:44:16.066213 24836 tablet_service.cc:1514] Processing DeleteTablet for tablet dad3a3348f054d0eacf69e0fb8d28c81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:16 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:57708
I20260811 01:44:16.067456 24671 tablet_service.cc:1514] Processing DeleteTablet for tablet dad3a3348f054d0eacf69e0fb8d28c81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:16 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:53420
I20260811 01:44:16.069538 24358 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:44:16.070472 24358 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:44:16.082360 24514 tablet_service.cc:1467] Processing CreateTablet for tablet bfaa84dde57d434598a025129958c0ca (DEFAULT_TABLE table=test-table [id=97ed62771bea45429239bc7ba100c7a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:16.082965 24514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfaa84dde57d434598a025129958c0ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:16.083045 24835 tablet_service.cc:1467] Processing CreateTablet for tablet bfaa84dde57d434598a025129958c0ca (DEFAULT_TABLE table=test-table [id=97ed62771bea45429239bc7ba100c7a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:16.083895 24835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfaa84dde57d434598a025129958c0ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:16.086457 24996 tablet_replica.cc:330] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c: stopping tablet replica
I20260811 01:44:16.086687 24669 tablet_service.cc:1467] Processing CreateTablet for tablet bfaa84dde57d434598a025129958c0ca (DEFAULT_TABLE table=test-table [id=97ed62771bea45429239bc7ba100c7a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:16.086963 24996 raft_consensus.cc:2229] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:44:16.087252 24669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfaa84dde57d434598a025129958c0ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:16.087396 24996 raft_consensus.cc:2258] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:16.096575 24944 tablet_bootstrap.cc:492] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93: Bootstrap starting.
I20260811 01:44:16.097016 24996 ts_tablet_manager.cc:1905] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:16.098891 24944 tablet_bootstrap.cc:654] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:16.101560 24940 tablet_bootstrap.cc:492] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460: Bootstrap starting.
I20260811 01:44:16.103533 24940 tablet_bootstrap.cc:654] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:16.108989 24941 tablet_bootstrap.cc:492] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c: Bootstrap starting.
I20260811 01:44:16.111073 24941 tablet_bootstrap.cc:654] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:16.114780 24944 tablet_bootstrap.cc:492] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93: No bootstrap required, opened a new log
I20260811 01:44:16.115026 24944 ts_tablet_manager.cc:1397] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.001s
I20260811 01:44:16.116174 24944 raft_consensus.cc:348] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:16.116530 24944 raft_consensus.cc:374] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:16.116650 24944 raft_consensus.cc:729] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd1b1cec455448b0ad426757b43c3b93, State: Initialized, Role: FOLLOWER
I20260811 01:44:16.117004 24944 consensus_queue.cc:260] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:16.120867 24996 ts_tablet_manager.cc:1918] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 01:44:16.121117 24996 log.cc:1196] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/dad3a3348f054d0eacf69e0fb8d28c81
I20260811 01:44:16.121795 24993 tablet_replica.cc:330] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460: stopping tablet replica
I20260811 01:44:16.121886 24996 ts_tablet_manager.cc:1939] T dad3a3348f054d0eacf69e0fb8d28c81 P d113db0ec3fb4739837c7c9bd832c37c: Deleting consensus metadata
I20260811 01:44:16.122391 24993 raft_consensus.cc:2229] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:44:16.121073 24944 ts_tablet_manager.cc:1428] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260811 01:44:16.126684 24995 tablet_replica.cc:330] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93: stopping tablet replica
I20260811 01:44:16.127271 24995 raft_consensus.cc:2229] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:44:16.127728 24995 raft_consensus.cc:2258] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:16.127883 24342 catalog_manager.cc:4841] TS d113db0ec3fb4739837c7c9bd832c37c (127.21.214.66:33981): tablet dad3a3348f054d0eacf69e0fb8d28c81 (table test-table [id=253fab3c49c24070bfb7787516a9bcef]) successfully deleted
I20260811 01:44:16.128472 24941 tablet_bootstrap.cc:492] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c: No bootstrap required, opened a new log
I20260811 01:44:16.128702 24941 ts_tablet_manager.cc:1397] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.000s
I20260811 01:44:16.128636 24993 raft_consensus.cc:2258] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:16.129688 24941 raft_consensus.cc:348] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:16.129987 24941 raft_consensus.cc:374] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:16.130080 24941 raft_consensus.cc:729] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d113db0ec3fb4739837c7c9bd832c37c, State: Initialized, Role: FOLLOWER
I20260811 01:44:16.130362 24941 consensus_queue.cc:260] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:16.131047 24941 ts_tablet_manager.cc:1428] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 01:44:16.131227 24993 ts_tablet_manager.cc:1905] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:16.131596 24940 tablet_bootstrap.cc:492] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460: No bootstrap required, opened a new log
I20260811 01:44:16.131798 24940 ts_tablet_manager.cc:1397] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.005s
I20260811 01:44:16.132771 24940 raft_consensus.cc:348] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:16.137573 24995 ts_tablet_manager.cc:1905] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:16.133088 24940 raft_consensus.cc:374] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:16.141289 24940 raft_consensus.cc:729] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d35f905fd45404998f016ad8d1a1460, State: Initialized, Role: FOLLOWER
I20260811 01:44:16.141697 24940 consensus_queue.cc:260] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:16.142657 24940 ts_tablet_manager.cc:1428] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460: Time spent starting tablet: real 0.011s user 0.000s sys 0.003s
I20260811 01:44:16.149823 24993 ts_tablet_manager.cc:1918] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 01:44:16.150071 24993 log.cc:1196] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/dad3a3348f054d0eacf69e0fb8d28c81
I20260811 01:44:16.153901 24993 ts_tablet_manager.cc:1939] T dad3a3348f054d0eacf69e0fb8d28c81 P 0d35f905fd45404998f016ad8d1a1460: Deleting consensus metadata
I20260811 01:44:16.156339 24343 catalog_manager.cc:4841] TS 0d35f905fd45404998f016ad8d1a1460 (127.21.214.65:44147): tablet dad3a3348f054d0eacf69e0fb8d28c81 (table test-table [id=253fab3c49c24070bfb7787516a9bcef]) successfully deleted
I20260811 01:44:16.156481 24995 ts_tablet_manager.cc:1918] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 01:44:16.156811 24995 log.cc:1196] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/dad3a3348f054d0eacf69e0fb8d28c81
I20260811 01:44:16.157619 24995 ts_tablet_manager.cc:1939] T dad3a3348f054d0eacf69e0fb8d28c81 P dd1b1cec455448b0ad426757b43c3b93: Deleting consensus metadata
I20260811 01:44:16.160030 24341 catalog_manager.cc:4841] TS dd1b1cec455448b0ad426757b43c3b93 (127.21.214.67:45639): tablet dad3a3348f054d0eacf69e0fb8d28c81 (table test-table [id=253fab3c49c24070bfb7787516a9bcef]) successfully deleted
I20260811 01:44:16.187350 24976 raft_consensus.cc:482] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:44:16.187606 24976 raft_consensus.cc:504] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:16.188441 24976 leader_election.cc:288] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d35f905fd45404998f016ad8d1a1460 (127.21.214.65:44147), d113db0ec3fb4739837c7c9bd832c37c (127.21.214.66:33981)
I20260811 01:44:16.189112 24537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfaa84dde57d434598a025129958c0ca" candidate_uuid: "dd1b1cec455448b0ad426757b43c3b93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d35f905fd45404998f016ad8d1a1460" is_pre_election: true
I20260811 01:44:16.189239 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfaa84dde57d434598a025129958c0ca" candidate_uuid: "dd1b1cec455448b0ad426757b43c3b93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d113db0ec3fb4739837c7c9bd832c37c" is_pre_election: true
I20260811 01:44:16.189443 24537 raft_consensus.cc:2454] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1b1cec455448b0ad426757b43c3b93 in term 0.
I20260811 01:44:16.189487 24693 raft_consensus.cc:2454] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1b1cec455448b0ad426757b43c3b93 in term 0.
I20260811 01:44:16.189993 24782 leader_election.cc:302] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d35f905fd45404998f016ad8d1a1460, dd1b1cec455448b0ad426757b43c3b93; no voters:
I20260811 01:44:16.190419 24976 raft_consensus.cc:2781] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:44:16.190560 24976 raft_consensus.cc:482] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:44:16.190686 24976 raft_consensus.cc:3037] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:16.194469 24976 raft_consensus.cc:504] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:16.195798 24537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfaa84dde57d434598a025129958c0ca" candidate_uuid: "dd1b1cec455448b0ad426757b43c3b93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d35f905fd45404998f016ad8d1a1460"
I20260811 01:44:16.196151 24537 raft_consensus.cc:3037] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:16.196169 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfaa84dde57d434598a025129958c0ca" candidate_uuid: "dd1b1cec455448b0ad426757b43c3b93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d113db0ec3fb4739837c7c9bd832c37c"
I20260811 01:44:16.196413 24693 raft_consensus.cc:3037] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:16.199750 24537 raft_consensus.cc:2454] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1b1cec455448b0ad426757b43c3b93 in term 1.
I20260811 01:44:16.199877 24693 raft_consensus.cc:2454] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1b1cec455448b0ad426757b43c3b93 in term 1.
I20260811 01:44:16.200623 24782 leader_election.cc:302] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d35f905fd45404998f016ad8d1a1460, dd1b1cec455448b0ad426757b43c3b93; no voters:
I20260811 01:44:16.200965 24965 raft_consensus.cc:2781] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:16.201637 24965 raft_consensus.cc:686] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [term 1 LEADER]: Becoming Leader. State: Replica: dd1b1cec455448b0ad426757b43c3b93, State: Running, Role: LEADER
I20260811 01:44:16.202100 24976 leader_election.cc:288] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [CANDIDATE]: Term 1 election: Requested vote from peers 0d35f905fd45404998f016ad8d1a1460 (127.21.214.65:44147), d113db0ec3fb4739837c7c9bd832c37c (127.21.214.66:33981)
I20260811 01:44:16.201979 24965 consensus_queue.cc:237] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:16.207187 24359 catalog_manager.cc:5495] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd1b1cec455448b0ad426757b43c3b93 (127.21.214.67). New cstate: current_term: 1 leader_uuid: "dd1b1cec455448b0ad426757b43c3b93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } health_report { overall_health: HEALTHY } } }
I20260811 01:44:16.254995 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:16.245461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57730 (local address 127.21.214.67:45639)
0811 01:44:16.245668 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:16.245674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:16.246264 (+ 590us) server_negotiation.cc:407] Connection header received
0811 01:44:16.246546 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:16.246552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:16.246640 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:16.246770 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:44:16.247258 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:16.248792 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:16.251796 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:16.252819 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:16.252980 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:16.253831 (+ 851us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:44:16.254042 (+ 211us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:44:16.254193 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:16.254403 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 01:44:16.254542 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 01:44:16.279551 24989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:16.270225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:45639 (local address 127.21.214.67:46963)
0811 01:44:16.270471 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:16.270494 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:16.270625 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:16.271402 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:16.271408 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:16.271440 (+ 32us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 01:44:16.271820 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:16.271835 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:16.273920 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:16.273926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:16.276404 (+ 2478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:16.276422 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:16.277873 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:16.277878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:16.278192 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:16.278248 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 01:44:16.278684 (+ 436us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 01:44:16.278699 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 01:44:16.278792 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 01:44:16.278967 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 01:44:16.280381 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:16.270824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:46963 (local address 127.21.214.67:45639)
0811 01:44:16.271050 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:16.271057 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:16.271075 (+ 18us) server_negotiation.cc:407] Connection header received
0811 01:44:16.271141 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:16.271146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:16.271223 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:16.271315 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:44:16.271999 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:16.273703 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:16.276589 (+ 2886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:16.277619 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:16.277750 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:16.278427 (+ 677us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:44:16.278544 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:44:16.278666 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:16.279916 (+ 1250us) server_negotiation.cc:299] Negotiation successful
0811 01:44:16.279999 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 01:44:16.301403 24537 raft_consensus.cc:1261] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460 [term 1 FOLLOWER]: Refusing update from remote peer dd1b1cec455448b0ad426757b43c3b93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:16.302073 24965 consensus_queue.cc:1035] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:16.303193 24693 raft_consensus.cc:1261] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c [term 1 FOLLOWER]: Refusing update from remote peer dd1b1cec455448b0ad426757b43c3b93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:16.303807 24965 consensus_queue.cc:1035] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:16.321053 24971 mvcc.cc:205] Tried to move back new op lower bound from 7317146240198676480 to 7317146239806312448. Current Snapshot: MvccSnapshot[applied={T|T < 7317146240198676480}]
W20260811 01:44:16.342654 24970 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 01:44:17.981758 24359 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:44:17.982092 24359 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:44:17.987814 24359 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268: Sending DeleteTablet for 3 replicas of tablet bfaa84dde57d434598a025129958c0ca
I20260811 01:44:17.988953 24516 tablet_service.cc:1514] Processing DeleteTablet for tablet bfaa84dde57d434598a025129958c0ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:17 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:53500
I20260811 01:44:17.988956 24836 tablet_service.cc:1514] Processing DeleteTablet for tablet bfaa84dde57d434598a025129958c0ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:17 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:57708
I20260811 01:44:17.989118 24669 tablet_service.cc:1514] Processing DeleteTablet for tablet bfaa84dde57d434598a025129958c0ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:17 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:53420
I20260811 01:44:18.002803 25027 tablet_replica.cc:330] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460: stopping tablet replica
I20260811 01:44:18.003353 25027 raft_consensus.cc:2229] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:44:18.003818 25027 raft_consensus.cc:2258] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:18.004889 25029 tablet_replica.cc:330] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c: stopping tablet replica
I20260811 01:44:18.005398 25027 ts_tablet_manager.cc:1905] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:18.005507 25029 raft_consensus.cc:2229] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:44:18.005983 25029 raft_consensus.cc:2258] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:18.007524 25029 ts_tablet_manager.cc:1905] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:18.010133 25028 tablet_replica.cc:330] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93: stopping tablet replica
I20260811 01:44:18.010455 25028 raft_consensus.cc:2229] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:44:18.011296 25028 raft_consensus.cc:2258] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:18.012545 25028 ts_tablet_manager.cc:1905] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:18.015362 25027 ts_tablet_manager.cc:1918] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 01:44:18.015655 25027 log.cc:1196] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/bfaa84dde57d434598a025129958c0ca
I20260811 01:44:18.016427 25027 ts_tablet_manager.cc:1939] T bfaa84dde57d434598a025129958c0ca P 0d35f905fd45404998f016ad8d1a1460: Deleting consensus metadata
I20260811 01:44:18.016714 25029 ts_tablet_manager.cc:1918] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 01:44:18.017011 25029 log.cc:1196] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/bfaa84dde57d434598a025129958c0ca
I20260811 01:44:18.017920 25029 ts_tablet_manager.cc:1939] T bfaa84dde57d434598a025129958c0ca P d113db0ec3fb4739837c7c9bd832c37c: Deleting consensus metadata
I20260811 01:44:18.018033 24343 catalog_manager.cc:4841] TS 0d35f905fd45404998f016ad8d1a1460 (127.21.214.65:44147): tablet bfaa84dde57d434598a025129958c0ca (table test-table [id=97ed62771bea45429239bc7ba100c7a0]) successfully deleted
I20260811 01:44:18.019392 24342 catalog_manager.cc:4841] TS d113db0ec3fb4739837c7c9bd832c37c (127.21.214.66:33981): tablet bfaa84dde57d434598a025129958c0ca (table test-table [id=97ed62771bea45429239bc7ba100c7a0]) successfully deleted
I20260811 01:44:18.021399 25028 ts_tablet_manager.cc:1918] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 01:44:18.021584 25028 log.cc:1196] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/bfaa84dde57d434598a025129958c0ca
I20260811 01:44:18.022102 25028 ts_tablet_manager.cc:1939] T bfaa84dde57d434598a025129958c0ca P dd1b1cec455448b0ad426757b43c3b93: Deleting consensus metadata
I20260811 01:44:18.023339 24341 catalog_manager.cc:4841] TS dd1b1cec455448b0ad426757b43c3b93 (127.21.214.67:45639): tablet bfaa84dde57d434598a025129958c0ca (table test-table [id=97ed62771bea45429239bc7ba100c7a0]) successfully deleted
Aug 11 01:44:18 dist-test-slave-9gcw krb5kdc[24307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412658, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 01:44:18 dist-test-slave-9gcw krb5kdc[24307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412658, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:18.072165 25039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:18.049003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49496 (local address 127.21.214.126:40057)
0811 01:44:18.050121 (+ 1118us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:18.050126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:18.050214 (+ 88us) server_negotiation.cc:407] Connection header received
0811 01:44:18.050280 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:18.050283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:18.050376 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:18.050476 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:18.052085 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:18.053325 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:18.054681 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:18.055249 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:18.055395 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:18.065421 (+ 10026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:18.065428 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:18.065432 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:18.065478 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:18.068845 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:18.069813 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:18.069818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:18.069820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:18.069965 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:18.070399 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:18.070402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:18.070405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:18.070882 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:18.071131 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:18.071422 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 01:44:18.071662 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":938,"thread_start_us":735,"threads_started":1}
I20260811 01:44:18.076184 24359 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:44:18.076943 24359 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:44:18.085048 24516 tablet_service.cc:1467] Processing CreateTablet for tablet a9ce679e1a8341f388344f572f56efca (DEFAULT_TABLE table=test-table [id=967c617dfd5b49e3ad2d3a4c444bb137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:18.085595 24516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9ce679e1a8341f388344f572f56efca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:18.085476 24669 tablet_service.cc:1467] Processing CreateTablet for tablet a9ce679e1a8341f388344f572f56efca (DEFAULT_TABLE table=test-table [id=967c617dfd5b49e3ad2d3a4c444bb137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:18.085779 24836 tablet_service.cc:1467] Processing CreateTablet for tablet a9ce679e1a8341f388344f572f56efca (DEFAULT_TABLE table=test-table [id=967c617dfd5b49e3ad2d3a4c444bb137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:18.086146 24669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9ce679e1a8341f388344f572f56efca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:18.086356 24836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9ce679e1a8341f388344f572f56efca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:18.092669 25041 tablet_bootstrap.cc:492] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460: Bootstrap starting.
I20260811 01:44:18.095785 25041 tablet_bootstrap.cc:654] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:18.096763 25043 tablet_bootstrap.cc:492] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c: Bootstrap starting.
I20260811 01:44:18.098315 25044 tablet_bootstrap.cc:492] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93: Bootstrap starting.
I20260811 01:44:18.099920 25043 tablet_bootstrap.cc:654] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:18.101112 25041 tablet_bootstrap.cc:492] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460: No bootstrap required, opened a new log
I20260811 01:44:18.101286 25044 tablet_bootstrap.cc:654] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:18.101429 25041 ts_tablet_manager.cc:1397] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 01:44:18.102741 25041 raft_consensus.cc:348] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:18.103032 25041 raft_consensus.cc:374] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:18.103166 25041 raft_consensus.cc:729] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d35f905fd45404998f016ad8d1a1460, State: Initialized, Role: FOLLOWER
I20260811 01:44:18.103497 25041 consensus_queue.cc:260] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:18.103837 25043 tablet_bootstrap.cc:492] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c: No bootstrap required, opened a new log
I20260811 01:44:18.104592 25041 ts_tablet_manager.cc:1428] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 01:44:18.104092 25043 ts_tablet_manager.cc:1397] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260811 01:44:18.106032 25043 raft_consensus.cc:348] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:18.106315 25043 raft_consensus.cc:374] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:18.106443 25043 raft_consensus.cc:729] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d113db0ec3fb4739837c7c9bd832c37c, State: Initialized, Role: FOLLOWER
I20260811 01:44:18.106750 25043 consensus_queue.cc:260] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:18.107997 25043 ts_tablet_manager.cc:1428] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 01:44:18.110360 25044 tablet_bootstrap.cc:492] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93: No bootstrap required, opened a new log
I20260811 01:44:18.110646 25044 ts_tablet_manager.cc:1397] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.006s
I20260811 01:44:18.111603 25044 raft_consensus.cc:348] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:18.111863 25044 raft_consensus.cc:374] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:18.112087 25044 raft_consensus.cc:729] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd1b1cec455448b0ad426757b43c3b93, State: Initialized, Role: FOLLOWER
I20260811 01:44:18.112421 25044 consensus_queue.cc:260] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:18.113131 25044 ts_tablet_manager.cc:1428] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 01:44:18.321684 25048 raft_consensus.cc:482] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:44:18.322032 25048 raft_consensus.cc:504] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:18.323119 25048 leader_election.cc:288] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d113db0ec3fb4739837c7c9bd832c37c (127.21.214.66:33981), dd1b1cec455448b0ad426757b43c3b93 (127.21.214.67:45639)
I20260811 01:44:18.323952 24864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9ce679e1a8341f388344f572f56efca" candidate_uuid: "0d35f905fd45404998f016ad8d1a1460" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd1b1cec455448b0ad426757b43c3b93" is_pre_election: true
I20260811 01:44:18.323997 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9ce679e1a8341f388344f572f56efca" candidate_uuid: "0d35f905fd45404998f016ad8d1a1460" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d113db0ec3fb4739837c7c9bd832c37c" is_pre_election: true
I20260811 01:44:18.324318 24864 raft_consensus.cc:2454] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d35f905fd45404998f016ad8d1a1460 in term 0.
I20260811 01:44:18.324405 24693 raft_consensus.cc:2454] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d35f905fd45404998f016ad8d1a1460 in term 0.
I20260811 01:44:18.325014 24457 leader_election.cc:302] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d35f905fd45404998f016ad8d1a1460, dd1b1cec455448b0ad426757b43c3b93; no voters:
I20260811 01:44:18.325484 25048 raft_consensus.cc:2781] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:44:18.325726 25048 raft_consensus.cc:482] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:44:18.325865 25048 raft_consensus.cc:3037] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:18.329344 25048 raft_consensus.cc:504] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:18.330190 25048 leader_election.cc:288] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [CANDIDATE]: Term 1 election: Requested vote from peers d113db0ec3fb4739837c7c9bd832c37c (127.21.214.66:33981), dd1b1cec455448b0ad426757b43c3b93 (127.21.214.67:45639)
I20260811 01:44:18.330925 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9ce679e1a8341f388344f572f56efca" candidate_uuid: "0d35f905fd45404998f016ad8d1a1460" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d113db0ec3fb4739837c7c9bd832c37c"
I20260811 01:44:18.330961 24864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9ce679e1a8341f388344f572f56efca" candidate_uuid: "0d35f905fd45404998f016ad8d1a1460" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd1b1cec455448b0ad426757b43c3b93"
I20260811 01:44:18.331172 24693 raft_consensus.cc:3037] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:18.331290 24864 raft_consensus.cc:3037] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:18.333709 24693 raft_consensus.cc:2454] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d35f905fd45404998f016ad8d1a1460 in term 1.
I20260811 01:44:18.334342 24458 leader_election.cc:302] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d35f905fd45404998f016ad8d1a1460, d113db0ec3fb4739837c7c9bd832c37c; no voters:
I20260811 01:44:18.334509 24864 raft_consensus.cc:2454] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d35f905fd45404998f016ad8d1a1460 in term 1.
I20260811 01:44:18.334838 25048 raft_consensus.cc:2781] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:18.335841 25048 raft_consensus.cc:686] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [term 1 LEADER]: Becoming Leader. State: Replica: 0d35f905fd45404998f016ad8d1a1460, State: Running, Role: LEADER
I20260811 01:44:18.336411 25048 consensus_queue.cc:237] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:18.340224 24358 catalog_manager.cc:5495] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d35f905fd45404998f016ad8d1a1460 (127.21.214.65). New cstate: current_term: 1 leader_uuid: "0d35f905fd45404998f016ad8d1a1460" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } health_report { overall_health: UNKNOWN } } }
I20260811 01:44:18.406415 25054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:18.399973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53518 (local address 127.21.214.65:44147)
0811 01:44:18.400942 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:18.400947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:18.400970 (+ 23us) server_negotiation.cc:407] Connection header received
0811 01:44:18.401023 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:18.401026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:18.401085 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:18.401282 (+ 197us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:44:18.401640 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:18.402626 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:18.403912 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:18.404499 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:18.404597 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:18.405039 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:44:18.405267 (+ 228us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:44:18.405450 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:18.405588 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 01:44:18.405650 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":842,"thread_start_us":683,"threads_started":1}
I20260811 01:44:18.417023 24864 raft_consensus.cc:1261] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93 [term 1 FOLLOWER]: Refusing update from remote peer 0d35f905fd45404998f016ad8d1a1460: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:18.417528 24693 raft_consensus.cc:1261] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c [term 1 FOLLOWER]: Refusing update from remote peer 0d35f905fd45404998f016ad8d1a1460: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:18.418119 25051 consensus_queue.cc:1035] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 01:44:18.418798 25048 consensus_queue.cc:1035] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:18.495800 24358 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:44:18.496199 24358 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:44:18.499727 24358 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268: Sending DeleteTablet for 3 replicas of tablet a9ce679e1a8341f388344f572f56efca
I20260811 01:44:18.500711 24516 tablet_service.cc:1514] Processing DeleteTablet for tablet a9ce679e1a8341f388344f572f56efca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:18 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:53500
I20260811 01:44:18.500787 24669 tablet_service.cc:1514] Processing DeleteTablet for tablet a9ce679e1a8341f388344f572f56efca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:18 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:53420
I20260811 01:44:18.501015 24836 tablet_service.cc:1514] Processing DeleteTablet for tablet a9ce679e1a8341f388344f572f56efca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:18 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:57708
I20260811 01:44:18.501214 25029 tablet_replica.cc:330] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c: stopping tablet replica
I20260811 01:44:18.501210 25027 tablet_replica.cc:330] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460: stopping tablet replica
I20260811 01:44:18.501480 25028 tablet_replica.cc:330] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93: stopping tablet replica
I20260811 01:44:18.501502 25029 raft_consensus.cc:2229] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:44:18.501483 25027 raft_consensus.cc:2229] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:44:18.501721 25028 raft_consensus.cc:2229] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:44:18.501734 25029 raft_consensus.cc:2258] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:18.501910 25027 raft_consensus.cc:2258] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:18.501945 25028 raft_consensus.cc:2258] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:18.502080 24358 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:44:18.502863 24358 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:44:18.503157 25029 ts_tablet_manager.cc:1905] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:18.503302 25028 ts_tablet_manager.cc:1905] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:18.503448 25027 ts_tablet_manager.cc:1905] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:18.511157 25028 ts_tablet_manager.cc:1918] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 01:44:18.511240 25029 ts_tablet_manager.cc:1918] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 01:44:18.511451 25029 log.cc:1196] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/a9ce679e1a8341f388344f572f56efca
I20260811 01:44:18.511487 25028 log.cc:1196] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/a9ce679e1a8341f388344f572f56efca
I20260811 01:44:18.512118 25029 ts_tablet_manager.cc:1939] T a9ce679e1a8341f388344f572f56efca P d113db0ec3fb4739837c7c9bd832c37c: Deleting consensus metadata
I20260811 01:44:18.512295 25027 ts_tablet_manager.cc:1918] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 01:44:18.512557 25027 log.cc:1196] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/a9ce679e1a8341f388344f572f56efca
I20260811 01:44:18.512876 24669 tablet_service.cc:1467] Processing CreateTablet for tablet 707bc99050684e64b05efb09ce0b14e7 (DEFAULT_TABLE table=test-table [id=a2fda63b68dc4cdfa88dd627a86f2c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:18.512918 24836 tablet_service.cc:1467] Processing CreateTablet for tablet 707bc99050684e64b05efb09ce0b14e7 (DEFAULT_TABLE table=test-table [id=a2fda63b68dc4cdfa88dd627a86f2c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:18.513216 25027 ts_tablet_manager.cc:1939] T a9ce679e1a8341f388344f572f56efca P 0d35f905fd45404998f016ad8d1a1460: Deleting consensus metadata
I20260811 01:44:18.513463 24669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707bc99050684e64b05efb09ce0b14e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:18.513749 24836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707bc99050684e64b05efb09ce0b14e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:18.514725 24343 catalog_manager.cc:4841] TS 0d35f905fd45404998f016ad8d1a1460 (127.21.214.65:44147): tablet a9ce679e1a8341f388344f572f56efca (table test-table [id=967c617dfd5b49e3ad2d3a4c444bb137]) successfully deleted
I20260811 01:44:18.515043 24516 tablet_service.cc:1467] Processing CreateTablet for tablet 707bc99050684e64b05efb09ce0b14e7 (DEFAULT_TABLE table=test-table [id=a2fda63b68dc4cdfa88dd627a86f2c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:18.515636 24516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707bc99050684e64b05efb09ce0b14e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:18.516893 24342 catalog_manager.cc:4841] TS d113db0ec3fb4739837c7c9bd832c37c (127.21.214.66:33981): tablet a9ce679e1a8341f388344f572f56efca (table test-table [id=967c617dfd5b49e3ad2d3a4c444bb137]) successfully deleted
I20260811 01:44:18.518779 25028 ts_tablet_manager.cc:1939] T a9ce679e1a8341f388344f572f56efca P dd1b1cec455448b0ad426757b43c3b93: Deleting consensus metadata
I20260811 01:44:18.522150 25043 tablet_bootstrap.cc:492] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c: Bootstrap starting.
I20260811 01:44:18.522742 24341 catalog_manager.cc:4841] TS dd1b1cec455448b0ad426757b43c3b93 (127.21.214.67:45639): tablet a9ce679e1a8341f388344f572f56efca (table test-table [id=967c617dfd5b49e3ad2d3a4c444bb137]) successfully deleted
I20260811 01:44:18.523383 25044 tablet_bootstrap.cc:492] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93: Bootstrap starting.
I20260811 01:44:18.524875 25043 tablet_bootstrap.cc:654] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:18.525574 25041 tablet_bootstrap.cc:492] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460: Bootstrap starting.
I20260811 01:44:18.526576 25044 tablet_bootstrap.cc:654] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:18.528378 25041 tablet_bootstrap.cc:654] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:18.529461 25043 tablet_bootstrap.cc:492] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c: No bootstrap required, opened a new log
I20260811 01:44:18.529695 25043 ts_tablet_manager.cc:1397] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260811 01:44:18.531149 25043 raft_consensus.cc:348] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:18.531404 25044 tablet_bootstrap.cc:492] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93: No bootstrap required, opened a new log
I20260811 01:44:18.531605 25041 tablet_bootstrap.cc:492] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460: No bootstrap required, opened a new log
I20260811 01:44:18.531611 25043 raft_consensus.cc:374] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:18.531713 25044 ts_tablet_manager.cc:1397] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260811 01:44:18.531801 25043 raft_consensus.cc:729] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d113db0ec3fb4739837c7c9bd832c37c, State: Initialized, Role: FOLLOWER
I20260811 01:44:18.531857 25041 ts_tablet_manager.cc:1397] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20260811 01:44:18.532480 25043 consensus_queue.cc:260] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:18.532655 25044 raft_consensus.cc:348] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:18.532990 25044 raft_consensus.cc:374] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:18.533185 25044 raft_consensus.cc:729] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd1b1cec455448b0ad426757b43c3b93, State: Initialized, Role: FOLLOWER
I20260811 01:44:18.533000 25041 raft_consensus.cc:348] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:18.533433 25041 raft_consensus.cc:374] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:18.533562 25041 raft_consensus.cc:729] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d35f905fd45404998f016ad8d1a1460, State: Initialized, Role: FOLLOWER
I20260811 01:44:18.533565 25044 consensus_queue.cc:260] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:18.533866 25041 consensus_queue.cc:260] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:18.534389 25043 ts_tablet_manager.cc:1428] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 01:44:18.535138 25044 ts_tablet_manager.cc:1428] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 01:44:18.535449 25041 ts_tablet_manager.cc:1428] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 01:44:18.557387 25061 raft_consensus.cc:482] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:44:18.557602 25061 raft_consensus.cc:504] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:18.558676 25061 leader_election.cc:288] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d113db0ec3fb4739837c7c9bd832c37c (127.21.214.66:33981), dd1b1cec455448b0ad426757b43c3b93 (127.21.214.67:45639)
I20260811 01:44:18.559487 24691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707bc99050684e64b05efb09ce0b14e7" candidate_uuid: "0d35f905fd45404998f016ad8d1a1460" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d113db0ec3fb4739837c7c9bd832c37c" is_pre_election: true
I20260811 01:44:18.559557 24864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707bc99050684e64b05efb09ce0b14e7" candidate_uuid: "0d35f905fd45404998f016ad8d1a1460" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd1b1cec455448b0ad426757b43c3b93" is_pre_election: true
I20260811 01:44:18.559854 24691 raft_consensus.cc:2454] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d35f905fd45404998f016ad8d1a1460 in term 0.
I20260811 01:44:18.560019 24864 raft_consensus.cc:2454] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d35f905fd45404998f016ad8d1a1460 in term 0.
I20260811 01:44:18.560700 24458 leader_election.cc:302] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d35f905fd45404998f016ad8d1a1460, d113db0ec3fb4739837c7c9bd832c37c; no voters:
I20260811 01:44:18.561200 25061 raft_consensus.cc:2781] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:44:18.561353 25061 raft_consensus.cc:482] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:44:18.561478 25061 raft_consensus.cc:3037] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:18.564682 25061 raft_consensus.cc:504] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:18.565344 25061 leader_election.cc:288] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [CANDIDATE]: Term 1 election: Requested vote from peers d113db0ec3fb4739837c7c9bd832c37c (127.21.214.66:33981), dd1b1cec455448b0ad426757b43c3b93 (127.21.214.67:45639)
I20260811 01:44:18.566025 24864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707bc99050684e64b05efb09ce0b14e7" candidate_uuid: "0d35f905fd45404998f016ad8d1a1460" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd1b1cec455448b0ad426757b43c3b93"
I20260811 01:44:18.566007 24691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707bc99050684e64b05efb09ce0b14e7" candidate_uuid: "0d35f905fd45404998f016ad8d1a1460" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d113db0ec3fb4739837c7c9bd832c37c"
I20260811 01:44:18.566251 24691 raft_consensus.cc:3037] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:18.566272 24864 raft_consensus.cc:3037] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:18.569024 24691 raft_consensus.cc:2454] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d35f905fd45404998f016ad8d1a1460 in term 1.
I20260811 01:44:18.569124 24864 raft_consensus.cc:2454] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d35f905fd45404998f016ad8d1a1460 in term 1.
I20260811 01:44:18.569566 24458 leader_election.cc:302] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d35f905fd45404998f016ad8d1a1460, d113db0ec3fb4739837c7c9bd832c37c; no voters:
I20260811 01:44:18.570012 25061 raft_consensus.cc:2781] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:18.570217 25061 raft_consensus.cc:686] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [term 1 LEADER]: Becoming Leader. State: Replica: 0d35f905fd45404998f016ad8d1a1460, State: Running, Role: LEADER
I20260811 01:44:18.570573 25061 consensus_queue.cc:237] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } }
I20260811 01:44:18.573940 24359 catalog_manager.cc:5495] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d35f905fd45404998f016ad8d1a1460 (127.21.214.65). New cstate: current_term: 1 leader_uuid: "0d35f905fd45404998f016ad8d1a1460" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d35f905fd45404998f016ad8d1a1460" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 44147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 } health_report { overall_health: UNKNOWN } } }
I20260811 01:44:18.613356 24693 raft_consensus.cc:1261] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c [term 1 FOLLOWER]: Refusing update from remote peer 0d35f905fd45404998f016ad8d1a1460: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:18.613998 25051 consensus_queue.cc:1035] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d113db0ec3fb4739837c7c9bd832c37c" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:18.614362 24864 raft_consensus.cc:1261] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93 [term 1 FOLLOWER]: Refusing update from remote peer 0d35f905fd45404998f016ad8d1a1460: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:18.615051 25051 consensus_queue.cc:1035] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1b1cec455448b0ad426757b43c3b93" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 01:44:18.642263 25063 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 01:44:20.240759 24357 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:44:20.241117 24357 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:44:20.247174 24357 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 926939ec8ecc414db40558db089b1268: Sending DeleteTablet for 3 replicas of tablet 707bc99050684e64b05efb09ce0b14e7
I20260811 01:44:20.248168 24516 tablet_service.cc:1514] Processing DeleteTablet for tablet 707bc99050684e64b05efb09ce0b14e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:20 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:53500
I20260811 01:44:20.248212 24669 tablet_service.cc:1514] Processing DeleteTablet for tablet 707bc99050684e64b05efb09ce0b14e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:20 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:53420
I20260811 01:44:20.248551 24836 tablet_service.cc:1514] Processing DeleteTablet for tablet 707bc99050684e64b05efb09ce0b14e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:20 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:57708
I20260811 01:44:20.249094 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 24440
I20260811 01:44:20.259555 25097 tablet_replica.cc:330] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460: stopping tablet replica
I20260811 01:44:20.260254 25098 tablet_replica.cc:330] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93: stopping tablet replica
I20260811 01:44:20.260895 25098 raft_consensus.cc:2229] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:44:20.261094 25097 raft_consensus.cc:2229] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:44:20.262336 25098 raft_consensus.cc:2258] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:20.262416 25097 raft_consensus.cc:2258] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:20.263913 25098 ts_tablet_manager.cc:1905] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:20.265738 25095 tablet_replica.cc:330] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c: stopping tablet replica
I20260811 01:44:20.266050 25095 raft_consensus.cc:2229] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:44:20.266412 25097 ts_tablet_manager.cc:1905] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:20.266498 25095 raft_consensus.cc:2258] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:20.267814 25095 ts_tablet_manager.cc:1905] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:20.273738 25098 ts_tablet_manager.cc:1918] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 01:44:20.274159 25098 log.cc:1196] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/707bc99050684e64b05efb09ce0b14e7
I20260811 01:44:20.275308 25098 ts_tablet_manager.cc:1939] T 707bc99050684e64b05efb09ce0b14e7 P dd1b1cec455448b0ad426757b43c3b93: Deleting consensus metadata
I20260811 01:44:20.275677 25097 ts_tablet_manager.cc:1918] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 01:44:20.275975 25097 log.cc:1196] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/707bc99050684e64b05efb09ce0b14e7
I20260811 01:44:20.276573 25097 ts_tablet_manager.cc:1939] T 707bc99050684e64b05efb09ce0b14e7 P 0d35f905fd45404998f016ad8d1a1460: Deleting consensus metadata
I20260811 01:44:20.277072 24341 catalog_manager.cc:4841] TS dd1b1cec455448b0ad426757b43c3b93 (127.21.214.67:45639): tablet 707bc99050684e64b05efb09ce0b14e7 (table test-table [id=a2fda63b68dc4cdfa88dd627a86f2c12]) successfully deleted
I20260811 01:44:20.277752 24343 catalog_manager.cc:4841] TS 0d35f905fd45404998f016ad8d1a1460 (127.21.214.65:44147): tablet 707bc99050684e64b05efb09ce0b14e7 (table test-table [id=a2fda63b68dc4cdfa88dd627a86f2c12]) successfully deleted
Aug 11 01:44:20 dist-test-slave-9gcw krb5kdc[24307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412658, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:44:20.284674 25095 ts_tablet_manager.cc:1918] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 01:44:20.284979 25095 log.cc:1196] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/707bc99050684e64b05efb09ce0b14e7
I20260811 01:44:20.285949 25095 ts_tablet_manager.cc:1939] T 707bc99050684e64b05efb09ce0b14e7 P d113db0ec3fb4739837c7c9bd832c37c: Deleting consensus metadata
I20260811 01:44:20.287400 24342 catalog_manager.cc:4841] TS d113db0ec3fb4739837c7c9bd832c37c (127.21.214.66:33981): tablet 707bc99050684e64b05efb09ce0b14e7 (table test-table [id=a2fda63b68dc4cdfa88dd627a86f2c12]) successfully deleted
I20260811 01:44:20.292480 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:20.250752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53534 (local address 127.21.214.65:44147)
0811 01:44:20.264333 (+ 13581us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:20.264344 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:20.264381 (+ 37us) server_negotiation.cc:407] Connection header received
0811 01:44:20.264472 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:20.264479 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:20.264602 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:20.264788 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:20.267616 (+ 2828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:20.269377 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:20.272085 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:20.273223 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:20.273389 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:20.281690 (+ 8301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:20.281713 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:20.281737 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:20.281789 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:20.287925 (+ 6136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:20.289030 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:20.289038 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:20.289041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:20.289335 (+ 294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:20.289857 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:20.289862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:20.289864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:20.290416 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:20.290886 (+ 470us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:20.291353 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 01:44:20.291659 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13389,"thread_start_us":14191,"threads_started":1}
W20260811 01:44:20.293407 24588 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53534
W20260811 01:44:20.294164 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 24440: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:44:20.294480 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 24440
I20260811 01:44:20.324538 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 24597
Aug 11 01:44:20 dist-test-slave-9gcw krb5kdc[24307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412658, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.214.66@KRBTEST.COM
I20260811 01:44:20.342298 25101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:20.325460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53436 (local address 127.21.214.66:33981)
0811 01:44:20.326046 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:20.326051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:20.326069 (+ 18us) server_negotiation.cc:407] Connection header received
0811 01:44:20.326127 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:20.326130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:20.326193 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:20.326291 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:20.327594 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:20.328359 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:20.329833 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:20.330247 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:20.330348 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:20.335528 (+ 5180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:20.335546 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:20.335564 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:20.335602 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:20.338921 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:20.339813 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:20.339821 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:20.339823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:20.339964 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:20.340359 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:20.340362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:20.340364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:20.340857 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:20.341088 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:20.341713 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 01:44:20.341936 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":317,"threads_started":1}
W20260811 01:44:20.343196 24748 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53436
W20260811 01:44:20.344133 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 24597: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:44:20.344355 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 24597
I20260811 01:44:20.370062 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 24760
Aug 11 01:44:20 dist-test-slave-9gcw krb5kdc[24307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412658, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.214.67@KRBTEST.COM
I20260811 01:44:20.387714 25102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:20.370972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57746 (local address 127.21.214.67:45639)
0811 01:44:20.372096 (+ 1124us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:20.372101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:20.372118 (+ 17us) server_negotiation.cc:407] Connection header received
0811 01:44:20.372175 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:20.372179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:20.372244 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:20.372385 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:20.373969 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:20.374750 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:20.376053 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:20.376487 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:20.376638 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:20.381368 (+ 4730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:20.381392 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:20.381419 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:20.381476 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:20.384726 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:20.385760 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:20.385768 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:20.385771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:20.385922 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:20.386281 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:20.386284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:20.386286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:20.386639 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:20.386937 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:20.387179 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 01:44:20.387361 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1003,"thread_start_us":813,"threads_started":1}
W20260811 01:44:20.388454 24916 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57746
W20260811 01:44:20.389132 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 24760: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:44:20.389369 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 24760
I20260811 01:44:20.420958 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 24324
I20260811 01:44:20.422401 24407 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:20.542547 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 24324
2026-08-11T01:44:20Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8902 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:44:20 dist-test-slave-9gcw krb5kdc[25106](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:44:20 dist-test-slave-9gcw krb5kdc[25106](info): set up 2 sockets
Aug 11 01:44:20 dist-test-slave-9gcw krb5kdc[25106](info): commencing operation
krb5kdc: starting...
W20260811 01:44:21.793243 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.004s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:44:21 dist-test-slave-9gcw krb5kdc[25106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:44:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:44:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:44:22.029448 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:45635
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:40275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:45635
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:22.390596 25120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:22.390908 25120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:22.390998 25120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:22.398499 25120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:44:22.398618 25120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:22.398665 25120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:22.398722 25120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:44:22.398769 25120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:44:22.408890 25120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:40275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:45635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:45635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:22.410308 25120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:22.411652 25120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:22.425779 25126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:22.425871 25129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:22.425828 25127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:22.426501 25120 server_base.cc:1034] running on GCE node
I20260811 01:44:22.427305 25120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:22.428819 25120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:22.430055 25120 hybrid_clock.cc:648] HybridClock initialized: now 1786412662430018 us; error 59 us; skew 500 ppm
I20260811 01:44:22.430440 25120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:22 dist-test-slave-9gcw krb5kdc[25106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:22.433719 25120 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:44:22.435273 25120 webserver.cc:460] Webserver started at http://127.21.214.126:39311/ using document root <none> and password file <none>
I20260811 01:44:22.436003 25120 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:22.436152 25120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:22.436491 25120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:22.439587 25120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "30540ed1413443b581fde150a467be34"
format_stamp: "Formatted at 2026-08-11 01:44:22 on dist-test-slave-9gcw"
server_key: "ad019134297da7a12dff6a1773a94bba"
server_key_iv: "5f10bd71d83bcefdb7c8cab70c7d6044"
server_key_version: "encryptionkey@0"
I20260811 01:44:22.440379 25120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "30540ed1413443b581fde150a467be34"
format_stamp: "Formatted at 2026-08-11 01:44:22 on dist-test-slave-9gcw"
server_key: "ad019134297da7a12dff6a1773a94bba"
server_key_iv: "5f10bd71d83bcefdb7c8cab70c7d6044"
server_key_version: "encryptionkey@0"
I20260811 01:44:22.447659 25120 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.005s
I20260811 01:44:22.451757 25136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:22.453508 25120 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 01:44:22.453713 25120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "30540ed1413443b581fde150a467be34"
format_stamp: "Formatted at 2026-08-11 01:44:22 on dist-test-slave-9gcw"
server_key: "ad019134297da7a12dff6a1773a94bba"
server_key_iv: "5f10bd71d83bcefdb7c8cab70c7d6044"
server_key_version: "encryptionkey@0"
I20260811 01:44:22.453939 25120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:22.488333 25120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:22.492799 25120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:22.493111 25120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:22.514585 25120 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:45635
I20260811 01:44:22.514629 25196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:45635 every 8 connection(s)
I20260811 01:44:22.516835 25120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:44:22.521879 25197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:22.524669 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 25120
I20260811 01:44:22.524895 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:44:22.525638 22361 external_mini_cluster.cc:1428] Setting key 872bbb1e03578d8b07d5403d59836190
I20260811 01:44:22.540174 25197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34: Bootstrap starting.
I20260811 01:44:22.544719 25197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:22.546205 25197 log.cc:824] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:22.550015 25197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34: No bootstrap required, opened a new log
Aug 11 01:44:22 dist-test-slave-9gcw krb5kdc[25106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:22.561352 25197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30540ed1413443b581fde150a467be34" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45635 } }
I20260811 01:44:22.561872 25197 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:22.561998 25197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30540ed1413443b581fde150a467be34, State: Initialized, Role: FOLLOWER
I20260811 01:44:22.562427 25201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:22.528826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52942 (local address 127.21.214.126:45635)
0811 01:44:22.535625 (+ 6799us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:22.535644 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:22.535715 (+ 71us) server_negotiation.cc:407] Connection header received
0811 01:44:22.538944 (+ 3229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:22.538980 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:22.539584 (+ 604us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:22.540150 (+ 566us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:22.541739 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:22.544085 (+ 2346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:22.545563 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:22.546946 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:22.547163 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:22.552359 (+ 5196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:22.552371 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:22.552393 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:22.552454 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:22.556841 (+ 4387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:22.557917 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:22.557927 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:22.557937 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:22.558182 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:22.558760 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:22.558771 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:22.558775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:22.559651 (+ 876us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:22.559968 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:22.560373 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 01:44:22.560843 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5610,"thread_start_us":5335,"threads_started":1}
I20260811 01:44:22.562748 25197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30540ed1413443b581fde150a467be34" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45635 } }
I20260811 01:44:22.563041 25197 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:44:22.563210 25197 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:44:22.563411 25197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:22.587301 25197 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30540ed1413443b581fde150a467be34" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45635 } }
I20260811 01:44:22.588140 25197 leader_election.cc:302] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30540ed1413443b581fde150a467be34; no voters:
I20260811 01:44:22.588968 25197 leader_election.cc:288] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:44:22.589113 25203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:22.590456 25203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34 [term 1 LEADER]: Becoming Leader. State: Replica: 30540ed1413443b581fde150a467be34, State: Running, Role: LEADER
I20260811 01:44:22.591248 25203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30540ed1413443b581fde150a467be34" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45635 } }
I20260811 01:44:22.591930 25197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:44:22.594686 25205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30540ed1413443b581fde150a467be34. Latest consensus state: current_term: 1 leader_uuid: "30540ed1413443b581fde150a467be34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30540ed1413443b581fde150a467be34" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45635 } } }
I20260811 01:44:22.595116 25205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34 [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:22.595434 25204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30540ed1413443b581fde150a467be34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30540ed1413443b581fde150a467be34" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45635 } } }
I20260811 01:44:22.595845 25204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34 [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:22.608498 25210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:44:22.615505 25210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:44:22.627403 25210 catalog_manager.cc:1347] Generated new cluster ID: 1eb2403a7c6445e18f34cca2d53d6e40
I20260811 01:44:22.627558 25210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:44:22.658337 25210 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:44:22.660782 25210 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:44:22.674381 25210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30540ed1413443b581fde150a467be34: Generated new TSK 0
I20260811 01:44:22.675397 25210 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
W20260811 01:44:23.664044 22361 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.21.214.65: real 0.972s user 0.002s sys 0.014s
I20260811 01:44:23.665123 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:45635
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:40275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:24.024550 25226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:24.024897 25226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:24.025094 25226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:24.032740 25226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:24.032883 25226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:24.033069 25226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:44:24.043668 25226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:40275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:45635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:24.045298 25226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:24.046774 25226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:24.061261 25233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:24.061475 25235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:24.061296 25232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:24.062450 25226 server_base.cc:1034] running on GCE node
I20260811 01:44:24.063213 25226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:24.064435 25226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:24.065730 25226 hybrid_clock.cc:648] HybridClock initialized: now 1786412664065650 us; error 58 us; skew 500 ppm
I20260811 01:44:24.066176 25226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:24 dist-test-slave-9gcw krb5kdc[25106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412664, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:24.078051 25226 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:44:24.079981 25226 webserver.cc:460] Webserver started at http://127.21.214.65:33507/ using document root <none> and password file <none>
I20260811 01:44:24.080826 25226 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:24.081012 25226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:24.081463 25226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:24.084961 25226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "d3cc210960544c3097dc36ed49944c43"
format_stamp: "Formatted at 2026-08-11 01:44:24 on dist-test-slave-9gcw"
server_key: "b05540efdbc64c03800be24f3b222ad8"
server_key_iv: "350cbe23871cec569b87342652f341f9"
server_key_version: "encryptionkey@0"
I20260811 01:44:24.086017 25226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "d3cc210960544c3097dc36ed49944c43"
format_stamp: "Formatted at 2026-08-11 01:44:24 on dist-test-slave-9gcw"
server_key: "b05540efdbc64c03800be24f3b222ad8"
server_key_iv: "350cbe23871cec569b87342652f341f9"
server_key_version: "encryptionkey@0"
I20260811 01:44:24.092208 25226 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 01:44:24.097366 25243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:24.099104 25226 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 01:44:24.099336 25226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "d3cc210960544c3097dc36ed49944c43"
format_stamp: "Formatted at 2026-08-11 01:44:24 on dist-test-slave-9gcw"
server_key: "b05540efdbc64c03800be24f3b222ad8"
server_key_iv: "350cbe23871cec569b87342652f341f9"
server_key_version: "encryptionkey@0"
I20260811 01:44:24.099594 25226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:24.115947 25226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:24.121693 25226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:24.122109 25226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:24.123229 25226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:24.125098 25226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:24.125238 25226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 01:44:24.125432 25226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:24.125522 25226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:24.166903 25226 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:36827
I20260811 01:44:24.166958 25379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:36827 every 8 connection(s)
I20260811 01:44:24.168506 25226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:44:24.170164 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 25226
I20260811 01:44:24.170442 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:44:24.171378 22361 external_mini_cluster.cc:1428] Setting key 9a7f6ac5f1ec6629aa21c865110800f2
Aug 11 01:44:24 dist-test-slave-9gcw krb5kdc[25106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412664, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:24.196398 25382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:24.171989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:50737 (local address 127.21.214.126:45635)
0811 01:44:24.172596 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:24.172603 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:24.173891 (+ 1288us) server_negotiation.cc:407] Connection header received
0811 01:44:24.176240 (+ 2349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:24.176249 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:24.176365 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:24.176505 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:24.178698 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:24.179525 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:24.181546 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:24.182057 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:24.182240 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:24.187151 (+ 4911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:24.187162 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:24.187171 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:24.187227 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:24.191651 (+ 4424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:24.192735 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:24.192740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:24.192742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:24.192996 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:24.193581 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:24.193586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:24.193589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:24.194071 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:24.194252 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:24.195737 (+ 1485us) server_negotiation.cc:299] Negotiation successful
0811 01:44:24.196003 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":295,"threads_started":1}
WARNING: no policy specified for kudu/127.21.214.66@KRBTEST.COM; defaulting to no policy
I20260811 01:44:24.197638 25383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:24.172713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:45635 (local address 127.21.214.65:50737)
0811 01:44:24.173711 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:24.173780 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:24.175859 (+ 2079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:24.176923 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:24.176933 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:24.177700 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:24.178459 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:24.178482 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:24.179743 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:24.179751 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:24.181362 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:24.181377 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:24.182343 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:24.182352 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:24.182639 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:24.183491 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:24.183518 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:24.186806 (+ 3288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:24.191897 (+ 5091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:24.191906 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:24.191922 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:24.192560 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:24.193120 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:24.193127 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:24.193132 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:24.193446 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:24.194227 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:24.194237 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:24.195253 (+ 1016us) client_negotiation.cc:769] Sending connection context
0811 01:44:24.195606 (+ 353us) client_negotiation.cc:240] Negotiation successful
0811 01:44:24.196225 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":434,"threads_started":1}
I20260811 01:44:24.199376 25380 heartbeater.cc:347] Connected to a master server at 127.21.214.126:45635
I20260811 01:44:24.199827 25380 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:24.200587 25380 heartbeater.cc:511] Master 127.21.214.126:45635 requested a full tablet report, sending...
I20260811 01:44:24.203192 25154 ts_manager.cc:194] Registered new tserver with Master: d3cc210960544c3097dc36ed49944c43 (127.21.214.65:36827)
I20260811 01:44:24.205252 25154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:50737
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
W20260811 01:44:24.369685 22361 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.21.214.66: real 0.195s user 0.002s sys 0.013s
I20260811 01:44:24.370604 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:45635
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:40275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:24.721417 25386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:24.721732 25386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:24.721943 25386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:24.729635 25386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:24.729758 25386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:24.729984 25386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:44:24.740254 25386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:40275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:45635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:24.741677 25386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:24.743100 25386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:24.757112 25392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:24.757238 25391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:24.758525 25394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:24.759176 25386 server_base.cc:1034] running on GCE node
I20260811 01:44:24.759871 25386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:24.760907 25386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:24.762178 25386 hybrid_clock.cc:648] HybridClock initialized: now 1786412664762094 us; error 60 us; skew 500 ppm
I20260811 01:44:24.762529 25386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:24 dist-test-slave-9gcw krb5kdc[25106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412664, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:24.766160 25386 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:44:24.767879 25386 webserver.cc:460] Webserver started at http://127.21.214.66:38217/ using document root <none> and password file <none>
I20260811 01:44:24.768654 25386 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:24.768808 25386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:24.769220 25386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:24.772368 25386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "a836a59c30f04c9fbcb65135e1aa239a"
format_stamp: "Formatted at 2026-08-11 01:44:24 on dist-test-slave-9gcw"
server_key: "b5561aa3ba6c554cae9160e96070608e"
server_key_iv: "0dfb5fffa3f770211f2170a3beb427d0"
server_key_version: "encryptionkey@0"
I20260811 01:44:24.773274 25386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "a836a59c30f04c9fbcb65135e1aa239a"
format_stamp: "Formatted at 2026-08-11 01:44:24 on dist-test-slave-9gcw"
server_key: "b5561aa3ba6c554cae9160e96070608e"
server_key_iv: "0dfb5fffa3f770211f2170a3beb427d0"
server_key_version: "encryptionkey@0"
I20260811 01:44:24.779014 25386 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 01:44:24.782824 25403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:24.784273 25386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 01:44:24.784482 25386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "a836a59c30f04c9fbcb65135e1aa239a"
format_stamp: "Formatted at 2026-08-11 01:44:24 on dist-test-slave-9gcw"
server_key: "b5561aa3ba6c554cae9160e96070608e"
server_key_iv: "0dfb5fffa3f770211f2170a3beb427d0"
server_key_version: "encryptionkey@0"
I20260811 01:44:24.784765 25386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:24.804812 25386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:24.809623 25386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:24.809957 25386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:24.811007 25386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:24.812474 25386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:24.812582 25386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:24.812752 25386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:24.812829 25386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:24.848724 25386 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:35329
I20260811 01:44:24.848799 25541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:35329 every 8 connection(s)
I20260811 01:44:24.850204 25386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:44:24.855738 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 25386
I20260811 01:44:24.855957 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:44:24.856554 22361 external_mini_cluster.cc:1428] Setting key 9f7c308990467f6684bb4ac34a5a4aa4
Aug 11 01:44:24 dist-test-slave-9gcw krb5kdc[25106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412664, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:24.873121 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:24.852991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:58597 (local address 127.21.214.126:45635)
0811 01:44:24.853514 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:24.853519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:24.854590 (+ 1071us) server_negotiation.cc:407] Connection header received
0811 01:44:24.856808 (+ 2218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:24.856814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:24.856911 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:24.857053 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:24.859351 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:24.860187 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:24.862036 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:24.862520 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:24.862694 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:24.866890 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:24.866897 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:24.866900 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:24.866937 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:24.869714 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:24.870606 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:24.870617 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:24.870620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:24.870755 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:24.871230 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:24.871234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:24.871236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:24.871525 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:24.871703 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:24.872465 (+ 762us) server_negotiation.cc:299] Negotiation successful
0811 01:44:24.872720 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":289,"threads_started":1}
I20260811 01:44:24.874225 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:24.853641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:45635 (local address 127.21.214.66:58597)
0811 01:44:24.854410 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:24.854459 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:24.856474 (+ 2015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:24.857474 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:24.857484 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:24.858460 (+ 976us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:24.859155 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:24.859176 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:24.860370 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:24.860381 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:24.861854 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:24.861868 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:24.862804 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:24.862813 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:24.863093 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:24.863901 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:24.863935 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:24.866507 (+ 2572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:24.869897 (+ 3390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:24.869908 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:24.869928 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:24.870458 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:24.870885 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:24.870890 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:24.870894 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:24.871096 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:24.871665 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:24.871675 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:24.872072 (+ 397us) client_negotiation.cc:769] Sending connection context
0811 01:44:24.872367 (+ 295us) client_negotiation.cc:240] Negotiation successful
0811 01:44:24.872953 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":297,"threads_started":1}
I20260811 01:44:24.875854 25542 heartbeater.cc:347] Connected to a master server at 127.21.214.126:45635
I20260811 01:44:24.876190 25542 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:24.876999 25542 heartbeater.cc:511] Master 127.21.214.126:45635 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.214.67@KRBTEST.COM; defaulting to no policy
I20260811 01:44:24.878707 25154 ts_manager.cc:194] Registered new tserver with Master: a836a59c30f04c9fbcb65135e1aa239a (127.21.214.66:35329)
I20260811 01:44:24.879585 25154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:58597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
I20260811 01:44:24.912871 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:45635
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:40275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:44:25.208897 25380 heartbeater.cc:503] Master 127.21.214.126:45635 was elected leader, sending a full tablet report...
W20260811 01:44:25.265893 25548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:25.266223 25548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:25.266323 25548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:25.273952 25548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:25.274082 25548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:25.274200 25548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:44:25.284358 25548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:40275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:45635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:25.285751 25548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:25.287145 25548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:25.301178 25557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:25.301299 25553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:25.301280 25555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:25.302364 25548 server_base.cc:1034] running on GCE node
I20260811 01:44:25.303054 25548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:25.304137 25548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:25.305346 25548 hybrid_clock.cc:648] HybridClock initialized: now 1786412665305269 us; error 71 us; skew 500 ppm
I20260811 01:44:25.305754 25548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:25 dist-test-slave-9gcw krb5kdc[25106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412665, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:25.309414 25548 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:44:25.311122 25548 webserver.cc:460] Webserver started at http://127.21.214.67:40927/ using document root <none> and password file <none>
I20260811 01:44:25.311875 25548 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:25.312027 25548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:25.312363 25548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:25.315560 25548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "a9d3a9000d4a4c3792758cd353f27950"
format_stamp: "Formatted at 2026-08-11 01:44:25 on dist-test-slave-9gcw"
server_key: "daeb8ce18ef43e2d2ff2b4c4ec67a49b"
server_key_iv: "844dbb24f013529e3abda08802b49ec5"
server_key_version: "encryptionkey@0"
I20260811 01:44:25.316397 25548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "a9d3a9000d4a4c3792758cd353f27950"
format_stamp: "Formatted at 2026-08-11 01:44:25 on dist-test-slave-9gcw"
server_key: "daeb8ce18ef43e2d2ff2b4c4ec67a49b"
server_key_iv: "844dbb24f013529e3abda08802b49ec5"
server_key_version: "encryptionkey@0"
I20260811 01:44:25.322078 25548 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 01:44:25.326061 25566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:25.327522 25548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:44:25.327718 25548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "a9d3a9000d4a4c3792758cd353f27950"
format_stamp: "Formatted at 2026-08-11 01:44:25 on dist-test-slave-9gcw"
server_key: "daeb8ce18ef43e2d2ff2b4c4ec67a49b"
server_key_iv: "844dbb24f013529e3abda08802b49ec5"
server_key_version: "encryptionkey@0"
I20260811 01:44:25.327930 25548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:25.344525 25548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:25.349119 25548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:25.349459 25548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:25.350471 25548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:25.351954 25548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:25.352067 25548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:25.352229 25548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:25.352303 25548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:25.387455 25548 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:43483
I20260811 01:44:25.387497 25705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:43483 every 8 connection(s)
I20260811 01:44:25.388993 25548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:44:25.390043 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 25548
I20260811 01:44:25.390261 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:44:25.390761 22361 external_mini_cluster.cc:1428] Setting key f0c1a6cba4de140705d89eeec64d8eb1
Aug 11 01:44:25 dist-test-slave-9gcw krb5kdc[25106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412665, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:25.411478 25708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:25.391877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:44973 (local address 127.21.214.126:45635)
0811 01:44:25.392363 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:25.392369 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:25.393639 (+ 1270us) server_negotiation.cc:407] Connection header received
0811 01:44:25.395826 (+ 2187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:25.395832 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:25.395921 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:25.396032 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:25.397739 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:25.398504 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:25.400130 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:25.400556 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:25.400706 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:25.404508 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:25.404513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:25.404517 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:25.404553 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:25.408287 (+ 3734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:25.409103 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:25.409108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:25.409110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:25.409312 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:25.409817 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:25.409821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:25.409823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:25.410119 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:25.410260 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:25.410984 (+ 724us) server_negotiation.cc:299] Negotiation successful
0811 01:44:25.411170 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":263,"threads_started":1}
I20260811 01:44:25.412557 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:25.392551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:45635 (local address 127.21.214.67:44973)
0811 01:44:25.393490 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:25.393535 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:25.395507 (+ 1972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:25.396300 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:25.396308 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:25.396868 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:25.397556 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:25.397574 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:25.398686 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:25.398692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:25.399977 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:25.399990 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:25.400799 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:25.400807 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:25.401050 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:25.401871 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:25.401901 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:25.404275 (+ 2374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:25.408438 (+ 4163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:25.408446 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:25.408462 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:25.408969 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:25.409415 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:25.409418 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:25.409421 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:25.409634 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:25.410240 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:25.410247 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:25.410589 (+ 342us) client_negotiation.cc:769] Sending connection context
0811 01:44:25.410882 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 01:44:25.411292 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":372,"threads_started":1}
I20260811 01:44:25.414373 25706 heartbeater.cc:347] Connected to a master server at 127.21.214.126:45635
I20260811 01:44:25.414732 25706 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:25.415540 25706 heartbeater.cc:511] Master 127.21.214.126:45635 requested a full tablet report, sending...
I20260811 01:44:25.417078 25154 ts_manager.cc:194] Registered new tserver with Master: a9d3a9000d4a4c3792758cd353f27950 (127.21.214.67:43483)
I20260811 01:44:25.418440 25154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:44973
I20260811 01:44:25.419965 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 01:44:25.459410 25708 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:44:25.456515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52950 (local address 127.21.214.126:45635)
0811 01:44:25.456887 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:25.456895 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:25.457254 (+ 359us) server_negotiation.cc:407] Connection header received
0811 01:44:25.457524 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:25.457528 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:25.457618 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:25.457734 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:25.459202 (+ 1468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52950: BlockingRecv error: recv got EOF from 127.0.0.1:52950 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 01:44:25.463476 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 25226
W20260811 01:44:25.466727 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 25226: Not authorized: Client connection negotiation failed: client connection to 127.21.214.65:36827: server requires authentication, but client does not have Kerberos credentials available
I20260811 01:44:25.466950 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 25226
I20260811 01:44:25.489420 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 25386
W20260811 01:44:25.492408 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 25386: Not authorized: Client connection negotiation failed: client connection to 127.21.214.66:35329: server requires authentication, but client does not have Kerberos credentials available
I20260811 01:44:25.492564 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 25386
I20260811 01:44:25.514360 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 25548
W20260811 01:44:25.517946 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 25548: Not authorized: Client connection negotiation failed: client connection to 127.21.214.67:43483: server requires authentication, but client does not have Kerberos credentials available
I20260811 01:44:25.518102 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 25548
I20260811 01:44:25.540100 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 25120
I20260811 01:44:25.541647 25189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:25.647382 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 25120
2026-08-11T01:44:25Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (5100 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:44:25 dist-test-slave-9gcw krb5kdc[25727](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:44:25 dist-test-slave-9gcw krb5kdc[25727](info): set up 2 sockets
Aug 11 01:44:25 dist-test-slave-9gcw krb5kdc[25727](info): commencing operation
krb5kdc: starting...
W20260811 01:44:26.885316 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:44:27 dist-test-slave-9gcw krb5kdc[25727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:44:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:44:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:44:27.133301 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:45603
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:33113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:45603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:27.482467 25741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:27.482864 25741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:27.483039 25741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:27.490594 25741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:44:27.490762 25741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:27.490881 25741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:27.490993 25741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:44:27.491077 25741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:44:27.501480 25741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:33113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:45603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:45603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:27.502956 25741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:27.504405 25741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:27.518221 25748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:27.518040 25747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:27.518010 25750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:27.519025 25741 server_base.cc:1034] running on GCE node
I20260811 01:44:27.519927 25741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:27.521517 25741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:27.522781 25741 hybrid_clock.cc:648] HybridClock initialized: now 1786412667522716 us; error 39 us; skew 500 ppm
I20260811 01:44:27.523167 25741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:27 dist-test-slave-9gcw krb5kdc[25727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412667, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:27.526728 25741 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:44:27.528452 25741 webserver.cc:460] Webserver started at http://127.21.214.126:41517/ using document root <none> and password file <none>
I20260811 01:44:27.529269 25741 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:27.529425 25741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:27.529791 25741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:27.533300 25741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "3445fbf3e55146868474cf2f07d2843a"
format_stamp: "Formatted at 2026-08-11 01:44:27 on dist-test-slave-9gcw"
server_key: "c00d0bb8947b98580e4056739073a003"
server_key_iv: "f6dca2dba69834ec41b7156eada3a0e7"
server_key_version: "encryptionkey@0"
I20260811 01:44:27.534214 25741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "3445fbf3e55146868474cf2f07d2843a"
format_stamp: "Formatted at 2026-08-11 01:44:27 on dist-test-slave-9gcw"
server_key: "c00d0bb8947b98580e4056739073a003"
server_key_iv: "f6dca2dba69834ec41b7156eada3a0e7"
server_key_version: "encryptionkey@0"
I20260811 01:44:27.539784 25741 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260811 01:44:27.543543 25760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:27.545043 25741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 01:44:27.545251 25741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "3445fbf3e55146868474cf2f07d2843a"
format_stamp: "Formatted at 2026-08-11 01:44:27 on dist-test-slave-9gcw"
server_key: "c00d0bb8947b98580e4056739073a003"
server_key_iv: "f6dca2dba69834ec41b7156eada3a0e7"
server_key_version: "encryptionkey@0"
I20260811 01:44:27.545463 25741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:27.568809 25741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:27.573278 25741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:27.573594 25741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:27.593946 25741 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:45603
I20260811 01:44:27.593981 25821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:45603 every 8 connection(s)
I20260811 01:44:27.595419 25741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:44:27.597322 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 25741
I20260811 01:44:27.597568 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:44:27.598414 22361 external_mini_cluster.cc:1428] Setting key ea272192be51b272246a7c59ba598a29
I20260811 01:44:27.600488 25822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 01:44:27 dist-test-slave-9gcw krb5kdc[25727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:27.614560 25822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a: Bootstrap starting.
I20260811 01:44:27.617877 25822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:27.619063 25822 log.cc:824] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:27.622707 25822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a: No bootstrap required, opened a new log
I20260811 01:44:27.623013 25827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:27.600378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55564 (local address 127.21.214.126:45603)
0811 01:44:27.601333 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:27.601345 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:27.601388 (+ 43us) server_negotiation.cc:407] Connection header received
0811 01:44:27.603083 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:27.603106 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:27.603499 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:27.603846 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:27.605730 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:27.607100 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:27.608896 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:27.609639 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:27.609794 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:27.614594 (+ 4800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:27.614601 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:27.614617 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:27.614654 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:27.617899 (+ 3245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:27.618698 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:27.618706 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:27.618713 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:27.618918 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:27.619373 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:27.619379 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:27.619382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:27.620201 (+ 819us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:27.620522 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:27.621005 (+ 483us) server_negotiation.cc:299] Negotiation successful
0811 01:44:27.621482 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":473,"threads_started":1}
I20260811 01:44:27.627588 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:44:27.634178 25822 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3445fbf3e55146868474cf2f07d2843a" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45603 } }
I20260811 01:44:27.634734 25822 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:27.634855 25822 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3445fbf3e55146868474cf2f07d2843a, State: Initialized, Role: FOLLOWER
I20260811 01:44:27.635636 25822 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3445fbf3e55146868474cf2f07d2843a" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45603 } }
I20260811 01:44:27.635962 25822 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:44:27.636109 25822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:44:27.636317 25822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:27.639222 25822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3445fbf3e55146868474cf2f07d2843a" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45603 } }
I20260811 01:44:27.639794 25822 leader_election.cc:302] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3445fbf3e55146868474cf2f07d2843a; no voters:
I20260811 01:44:27.640537 25822 leader_election.cc:288] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:44:27.640672 25830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:27.641906 25830 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a [term 1 LEADER]: Becoming Leader. State: Replica: 3445fbf3e55146868474cf2f07d2843a, State: Running, Role: LEADER
I20260811 01:44:27.642639 25830 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3445fbf3e55146868474cf2f07d2843a" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45603 } }
I20260811 01:44:27.643144 25822 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:44:27.645917 25831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3445fbf3e55146868474cf2f07d2843a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3445fbf3e55146868474cf2f07d2843a" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45603 } } }
I20260811 01:44:27.646370 25831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:27.650182 25834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3445fbf3e55146868474cf2f07d2843a. Latest consensus state: current_term: 1 leader_uuid: "3445fbf3e55146868474cf2f07d2843a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3445fbf3e55146868474cf2f07d2843a" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45603 } } }
I20260811 01:44:27.650632 25834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:27.652520 25837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:44:27.659420 25837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:44:27.668220 25851 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 01:44:27.671425 25837 catalog_manager.cc:1347] Generated new cluster ID: 51a371f1035f4ac6b527c4adc35d92e8
I20260811 01:44:27.671587 25837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:44:27.686816 25827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:27.675733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55576 (local address 127.21.214.126:45603)
0811 01:44:27.675901 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:27.675907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:27.677741 (+ 1834us) server_negotiation.cc:407] Connection header received
0811 01:44:27.677920 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:27.677926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:27.678022 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:27.678152 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:27.679364 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:27.680685 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:27.682683 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:27.683372 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:27.683533 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:27.684804 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:27.684811 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:27.684816 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:27.685805 (+ 989us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:27.685928 (+ 123us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:27.685977 (+ 49us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:27.686104 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:27.686285 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 01:44:27.686396 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 01:44:27.688686 25837 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:44:27.690160 25837 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:44:27.703907 25837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3445fbf3e55146868474cf2f07d2843a: Generated new TSK 0
I20260811 01:44:27.704797 25837 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:44:27.716595 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 25741
I20260811 01:44:27.717878 25817 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:27.811493 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 25741
2026-08-11T01:44:27Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2157 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:44:27 dist-test-slave-9gcw krb5kdc[25862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:44:27 dist-test-slave-9gcw krb5kdc[25862](info): set up 2 sockets
Aug 11 01:44:27 dist-test-slave-9gcw krb5kdc[25862](info): commencing operation
krb5kdc: starting...
W20260811 01:44:29.077848 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:44:29 dist-test-slave-9gcw krb5kdc[25862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:44:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:44:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:44:29.318331 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:33365
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:33365
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:29.669170 25878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:29.669576 25878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:29.669760 25878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:29.677203 25878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:44:29.677384 25878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:29.677508 25878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:29.677618 25878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:44:29.677716 25878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:44:29.688125 25878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:33365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:33365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:29.689778 25878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:29.691200 25878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:29.703102 25883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:29.703055 25884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:29.703141 25886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:29.704420 25878 server_base.cc:1034] running on GCE node
I20260811 01:44:29.705437 25878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:29.706811 25878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:29.708027 25878 hybrid_clock.cc:648] HybridClock initialized: now 1786412669707984 us; error 27 us; skew 500 ppm
I20260811 01:44:29.708333 25878 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:29 dist-test-slave-9gcw krb5kdc[25862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412669, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:29.716475 25878 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:44:29.718398 25878 webserver.cc:460] Webserver started at http://127.21.214.126:40171/ using document root <none> and password file <none>
I20260811 01:44:29.719190 25878 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:29.719359 25878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:29.719732 25878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:29.723248 25878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "f4431c09d113487da85a44a34a1a798f"
format_stamp: "Formatted at 2026-08-11 01:44:29 on dist-test-slave-9gcw"
server_key: "8271a9ae379c052c97f981e44dcbf6c1"
server_key_iv: "334b11658969c570cbbe7e1a16b31a3b"
server_key_version: "encryptionkey@0"
I20260811 01:44:29.724121 25878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "f4431c09d113487da85a44a34a1a798f"
format_stamp: "Formatted at 2026-08-11 01:44:29 on dist-test-slave-9gcw"
server_key: "8271a9ae379c052c97f981e44dcbf6c1"
server_key_iv: "334b11658969c570cbbe7e1a16b31a3b"
server_key_version: "encryptionkey@0"
I20260811 01:44:29.730172 25878 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 01:44:29.733924 25897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:29.735399 25878 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:44:29.735591 25878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "f4431c09d113487da85a44a34a1a798f"
format_stamp: "Formatted at 2026-08-11 01:44:29 on dist-test-slave-9gcw"
server_key: "8271a9ae379c052c97f981e44dcbf6c1"
server_key_iv: "334b11658969c570cbbe7e1a16b31a3b"
server_key_version: "encryptionkey@0"
I20260811 01:44:29.735869 25878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:29.758050 25878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:29.762156 25878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:29.762458 25878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:29.782485 25878 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:33365
I20260811 01:44:29.782545 25958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:33365 every 8 connection(s)
I20260811 01:44:29.783921 25878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:44:29.787853 25959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:29.793108 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 25878
I20260811 01:44:29.793401 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:44:29.794085 22361 external_mini_cluster.cc:1428] Setting key a85b83841db62f06bdd3abce67e1dceb
I20260811 01:44:29.799163 25959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f: Bootstrap starting.
I20260811 01:44:29.802968 25959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:29.804025 25959 log.cc:824] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:29.807898 25959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f: No bootstrap required, opened a new log
Aug 11 01:44:29 dist-test-slave-9gcw krb5kdc[25862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:29.819088 25959 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4431c09d113487da85a44a34a1a798f" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33365 } }
I20260811 01:44:29.819557 25963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:29.796279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43518 (local address 127.21.214.126:33365)
0811 01:44:29.797295 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:29.797307 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:29.797348 (+ 41us) server_negotiation.cc:407] Connection header received
0811 01:44:29.798971 (+ 1623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:29.798994 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:29.799414 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:29.799753 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:29.801615 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:29.802960 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:29.804245 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:29.805318 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:29.805529 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:29.810308 (+ 4779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:29.810324 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:29.810350 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:29.810403 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:29.814618 (+ 4215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:29.815353 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:29.815364 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:29.815379 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:29.815596 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:29.815961 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:29.815966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:29.815970 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:29.816757 (+ 787us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:29.817123 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:29.817625 (+ 502us) server_negotiation.cc:299] Negotiation successful
0811 01:44:29.818131 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":763,"thread_start_us":532,"threads_started":1}
I20260811 01:44:29.819589 25959 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:29.819692 25959 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4431c09d113487da85a44a34a1a798f, State: Initialized, Role: FOLLOWER
I20260811 01:44:29.820292 25959 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4431c09d113487da85a44a34a1a798f" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33365 } }
I20260811 01:44:29.820534 25959 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:44:29.820706 25959 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:44:29.820896 25959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:29.823350 25959 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4431c09d113487da85a44a34a1a798f" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33365 } }
I20260811 01:44:29.823861 25959 leader_election.cc:302] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4431c09d113487da85a44a34a1a798f; no voters:
I20260811 01:44:29.824551 25959 leader_election.cc:288] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:44:29.824739 25966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:29.826534 25959 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:44:29.828925 25966 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f [term 1 LEADER]: Becoming Leader. State: Replica: f4431c09d113487da85a44a34a1a798f, State: Running, Role: LEADER
I20260811 01:44:29.829548 25966 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4431c09d113487da85a44a34a1a798f" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33365 } }
I20260811 01:44:29.832618 25967 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4431c09d113487da85a44a34a1a798f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4431c09d113487da85a44a34a1a798f" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33365 } } }
I20260811 01:44:29.833009 25967 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:29.833232 25968 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4431c09d113487da85a44a34a1a798f. Latest consensus state: current_term: 1 leader_uuid: "f4431c09d113487da85a44a34a1a798f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4431c09d113487da85a44a34a1a798f" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33365 } } }
I20260811 01:44:29.833642 25968 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:29.834944 25976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:44:29.839841 25976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:44:29.848482 25976 catalog_manager.cc:1347] Generated new cluster ID: 437ac254df594e14b75871e1eb72e26e
I20260811 01:44:29.848603 25976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:44:29.860543 25976 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:44:29.862036 25976 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:44:29.869730 25976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4431c09d113487da85a44a34a1a798f: Generated new TSK 0
I20260811 01:44:29.870833 25976 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
I20260811 01:44:29.937196 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:33365
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:30.287077 25989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:30.287382 25989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:30.287480 25989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:30.295053 25989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:30.295184 25989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:30.295302 25989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:44:30.305387 25989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:33365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:30.307132 25989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:30.308651 25989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:30.323812 25996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:30.323836 25995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:30.324470 25998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:30.324864 25989 server_base.cc:1034] running on GCE node
I20260811 01:44:30.325713 25989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:30.326946 25989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:30.328253 25989 hybrid_clock.cc:648] HybridClock initialized: now 1786412670328179 us; error 62 us; skew 500 ppm
I20260811 01:44:30.328570 25989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:30 dist-test-slave-9gcw krb5kdc[25862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412670, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:30.332382 25989 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:44:30.334364 25989 webserver.cc:460] Webserver started at http://127.21.214.65:36877/ using document root <none> and password file <none>
I20260811 01:44:30.335206 25989 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:30.335317 25989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:30.335629 25989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:30.338932 25989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "4037a1010b134f46a99f5a2035eb303d"
format_stamp: "Formatted at 2026-08-11 01:44:30 on dist-test-slave-9gcw"
server_key: "db9bdcebc3c9f866342f862ab0510cc4"
server_key_iv: "5e3cd45a32a1b3b263b16df5f9407a5c"
server_key_version: "encryptionkey@0"
I20260811 01:44:30.339814 25989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "4037a1010b134f46a99f5a2035eb303d"
format_stamp: "Formatted at 2026-08-11 01:44:30 on dist-test-slave-9gcw"
server_key: "db9bdcebc3c9f866342f862ab0510cc4"
server_key_iv: "5e3cd45a32a1b3b263b16df5f9407a5c"
server_key_version: "encryptionkey@0"
I20260811 01:44:30.345456 25989 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 01:44:30.349200 26005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:30.350878 25989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:44:30.351102 25989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "4037a1010b134f46a99f5a2035eb303d"
format_stamp: "Formatted at 2026-08-11 01:44:30 on dist-test-slave-9gcw"
server_key: "db9bdcebc3c9f866342f862ab0510cc4"
server_key_iv: "5e3cd45a32a1b3b263b16df5f9407a5c"
server_key_version: "encryptionkey@0"
I20260811 01:44:30.351331 25989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:30.378064 25989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:30.382918 25989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:30.383257 25989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:30.384289 25989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:30.385797 25989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:30.385903 25989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:30.386061 25989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:30.386138 25989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:30.421300 25989 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:46295
I20260811 01:44:30.421387 26136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:46295 every 8 connection(s)
I20260811 01:44:30.422637 25989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:44:30.431627 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 25989
I20260811 01:44:30.431852 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:44:30.432677 22361 external_mini_cluster.cc:1428] Setting key f1b1f6c1e9e3d24c1e05ac009a7b26ee
Aug 11 01:44:30 dist-test-slave-9gcw krb5kdc[25862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412670, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:30.445464 26139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:30.425303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:45209 (local address 127.21.214.126:33365)
0811 01:44:30.425761 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:30.425766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:30.426757 (+ 991us) server_negotiation.cc:407] Connection header received
0811 01:44:30.428955 (+ 2198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:30.428961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:30.429080 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:30.429259 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:30.430874 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:30.432040 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:30.434221 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:30.434746 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:30.434851 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:30.438704 (+ 3853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:30.438713 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:30.438720 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:30.438776 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:30.442155 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:30.442971 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:30.442977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:30.442981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:30.443117 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:30.443574 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:30.443579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:30.443582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:30.443885 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:30.444062 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:30.444802 (+ 740us) server_negotiation.cc:299] Negotiation successful
0811 01:44:30.445038 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":249,"threads_started":1}
I20260811 01:44:30.446585 26140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:30.425788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:33365 (local address 127.21.214.65:45209)
0811 01:44:30.426557 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:30.426605 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:30.428647 (+ 2042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:30.429518 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:30.429528 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:30.430071 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:30.430677 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:30.430695 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:30.432248 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:30.432252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:30.434062 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:30.434079 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:30.435017 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:30.435024 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:30.435247 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:30.436037 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:30.436060 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:30.438407 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:30.442318 (+ 3911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:30.442329 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:30.442348 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:30.442832 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:30.443241 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:30.443246 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:30.443250 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:30.443447 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:30.444031 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:30.444040 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:30.444406 (+ 366us) client_negotiation.cc:769] Sending connection context
0811 01:44:30.444687 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 01:44:30.445298 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":316,"threads_started":1}
I20260811 01:44:30.448462 26137 heartbeater.cc:347] Connected to a master server at 127.21.214.126:33365
I20260811 01:44:30.448828 26137 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:30.449666 26137 heartbeater.cc:511] Master 127.21.214.126:33365 requested a full tablet report, sending...
I20260811 01:44:30.451972 25918 ts_manager.cc:194] Registered new tserver with Master: 4037a1010b134f46a99f5a2035eb303d (127.21.214.65:46295)
I20260811 01:44:30.453735 25918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:45209
WARNING: no policy specified for kudu/127.21.214.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
I20260811 01:44:30.488945 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:33365
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:30.842636 26143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:30.842947 26143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:30.843042 26143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:30.850404 26143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:30.850528 26143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:30.850648 26143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:44:30.860913 26143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:33365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:30.862310 26143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:30.863631 26143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:30.878539 26149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:30.878587 26151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:30.878549 26148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:30.879421 26143 server_base.cc:1034] running on GCE node
I20260811 01:44:30.880205 26143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:30.881438 26143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:30.882928 26143 hybrid_clock.cc:648] HybridClock initialized: now 1786412670882870 us; error 63 us; skew 500 ppm
I20260811 01:44:30.883337 26143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:30 dist-test-slave-9gcw krb5kdc[25862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412670, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:30.887223 26143 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:44:30.888801 26143 webserver.cc:460] Webserver started at http://127.21.214.66:37875/ using document root <none> and password file <none>
I20260811 01:44:30.889628 26143 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:30.889757 26143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:30.890057 26143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:30.893381 26143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "e383a762e023483fade78ab54a4861a8"
format_stamp: "Formatted at 2026-08-11 01:44:30 on dist-test-slave-9gcw"
server_key: "fb726b52c02494f113f123745caad2a0"
server_key_iv: "fa4f4dd8fe2966bb56d74895778aa4ea"
server_key_version: "encryptionkey@0"
I20260811 01:44:30.894534 26143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "e383a762e023483fade78ab54a4861a8"
format_stamp: "Formatted at 2026-08-11 01:44:30 on dist-test-slave-9gcw"
server_key: "fb726b52c02494f113f123745caad2a0"
server_key_iv: "fa4f4dd8fe2966bb56d74895778aa4ea"
server_key_version: "encryptionkey@0"
I20260811 01:44:30.900673 26143 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 01:44:30.904589 26161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:30.906206 26143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:44:30.906406 26143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "e383a762e023483fade78ab54a4861a8"
format_stamp: "Formatted at 2026-08-11 01:44:30 on dist-test-slave-9gcw"
server_key: "fb726b52c02494f113f123745caad2a0"
server_key_iv: "fa4f4dd8fe2966bb56d74895778aa4ea"
server_key_version: "encryptionkey@0"
I20260811 01:44:30.906623 26143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:30.947424 26143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:30.953213 26143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:30.953581 26143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:30.954676 26143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:30.956169 26143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:30.956252 26143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:30.956426 26143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:30.956506 26143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:30.993280 26143 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:37263
I20260811 01:44:30.993396 26299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:37263 every 8 connection(s)
I20260811 01:44:30.994823 26143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:44:31.004138 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 26143
I20260811 01:44:31.004338 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:44:31.004849 22361 external_mini_cluster.cc:1428] Setting key d1584178ea0ebedb39db095e7680f88a
Aug 11 01:44:31 dist-test-slave-9gcw krb5kdc[25862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412670, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:31.019905 26302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:30.997648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:49933 (local address 127.21.214.126:33365)
0811 01:44:30.998132 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:30.998139 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:30.999322 (+ 1183us) server_negotiation.cc:407] Connection header received
0811 01:44:31.001461 (+ 2139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:31.001466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:31.001543 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:31.001648 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:31.003358 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:31.004399 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:31.006498 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:31.007167 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:31.007282 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:31.012413 (+ 5131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:31.012422 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:31.012431 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:31.012484 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:31.016245 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:31.017338 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:31.017344 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:31.017347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:31.017517 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:31.018068 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:31.018071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:31.018073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:31.018344 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:31.018457 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:31.019405 (+ 948us) server_negotiation.cc:299] Negotiation successful
0811 01:44:31.019611 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":212,"threads_started":1}
I20260811 01:44:31.021615 26305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:30.998323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:33365 (local address 127.21.214.66:49933)
0811 01:44:30.999130 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:30.999181 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:31.001067 (+ 1886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:31.001905 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:31.001914 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:31.002524 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:31.003174 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:31.003194 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:31.004578 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:31.004583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:31.006339 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:31.006353 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:31.007431 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:31.007440 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:31.007926 (+ 486us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:31.008740 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:31.008782 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:31.012094 (+ 3312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:31.016420 (+ 4326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:31.016434 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:31.016455 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:31.017187 (+ 732us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:31.017645 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:31.017653 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:31.017659 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:31.017949 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:31.018478 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:31.018489 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:31.018996 (+ 507us) client_negotiation.cc:769] Sending connection context
0811 01:44:31.019335 (+ 339us) client_negotiation.cc:240] Negotiation successful
0811 01:44:31.019956 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":313,"threads_started":1}
I20260811 01:44:31.023216 26300 heartbeater.cc:347] Connected to a master server at 127.21.214.126:33365
I20260811 01:44:31.023550 26300 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:31.024341 26300 heartbeater.cc:511] Master 127.21.214.126:33365 requested a full tablet report, sending...
I20260811 01:44:31.026149 25918 ts_manager.cc:194] Registered new tserver with Master: e383a762e023483fade78ab54a4861a8 (127.21.214.66:37263)
I20260811 01:44:31.027098 25918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:49933
WARNING: no policy specified for kudu/127.21.214.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
I20260811 01:44:31.062604 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:33365
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:31.428036 26308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:31.428383 26308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:31.428573 26308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:31.437390 26308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:31.437572 26308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:31.437772 26308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:44:31.449457 26308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:33365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:31.450987 26308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:31.452466 26308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 01:44:31.457410 26137 heartbeater.cc:503] Master 127.21.214.126:33365 was elected leader, sending a full tablet report...
W20260811 01:44:31.468576 26314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:31.468611 26316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:31.468663 26313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:31.469287 26308 server_base.cc:1034] running on GCE node
I20260811 01:44:31.470144 26308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:31.471243 26308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:31.472518 26308 hybrid_clock.cc:648] HybridClock initialized: now 1786412671472443 us; error 63 us; skew 500 ppm
I20260811 01:44:31.472883 26308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:31 dist-test-slave-9gcw krb5kdc[25862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412671, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:31.476329 26308 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:44:31.478008 26308 webserver.cc:460] Webserver started at http://127.21.214.67:36753/ using document root <none> and password file <none>
I20260811 01:44:31.478782 26308 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:31.478899 26308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:31.479333 26308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:31.482695 26308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "63e53e93f2504952bbd6da369929fd61"
format_stamp: "Formatted at 2026-08-11 01:44:31 on dist-test-slave-9gcw"
server_key: "c9c00406e339436828f608852f2c05da"
server_key_iv: "c8d09fe527bfedd9c749886b7d4eb1da"
server_key_version: "encryptionkey@0"
I20260811 01:44:31.483565 26308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "63e53e93f2504952bbd6da369929fd61"
format_stamp: "Formatted at 2026-08-11 01:44:31 on dist-test-slave-9gcw"
server_key: "c9c00406e339436828f608852f2c05da"
server_key_iv: "c8d09fe527bfedd9c749886b7d4eb1da"
server_key_version: "encryptionkey@0"
I20260811 01:44:31.489374 26308 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 01:44:31.493548 26326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:31.495088 26308 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 01:44:31.495314 26308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "63e53e93f2504952bbd6da369929fd61"
format_stamp: "Formatted at 2026-08-11 01:44:31 on dist-test-slave-9gcw"
server_key: "c9c00406e339436828f608852f2c05da"
server_key_iv: "c8d09fe527bfedd9c749886b7d4eb1da"
server_key_version: "encryptionkey@0"
I20260811 01:44:31.495576 26308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:31.523290 26308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:31.528306 26308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:31.528625 26308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:31.529749 26308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:31.531255 26308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:31.531373 26308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:31.531528 26308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:31.531625 26308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:31.567855 26308 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:40653
I20260811 01:44:31.567893 26467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:40653 every 8 connection(s)
I20260811 01:44:31.569268 26308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:44:31.577934 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 26308
I20260811 01:44:31.578136 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:44:31.578706 22361 external_mini_cluster.cc:1428] Setting key e3ea2e2cc913694202dc22af05062ff0
Aug 11 01:44:31 dist-test-slave-9gcw krb5kdc[25862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412671, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:31.591019 26470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:31.572047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:41439 (local address 127.21.214.126:33365)
0811 01:44:31.572520 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:31.572525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:31.573498 (+ 973us) server_negotiation.cc:407] Connection header received
0811 01:44:31.575410 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:31.575416 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:31.575527 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:31.575657 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:31.577269 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:31.578454 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:31.580623 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:31.581227 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:31.581374 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:31.585200 (+ 3826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:31.585207 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:31.585212 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:31.585257 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:31.587732 (+ 2475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:31.588523 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:31.588530 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:31.588533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:31.588667 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:31.589118 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:31.589123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:31.589127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:31.589453 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:31.589629 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:31.590364 (+ 735us) server_negotiation.cc:299] Negotiation successful
0811 01:44:31.590597 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":255,"threads_started":1}
I20260811 01:44:31.592047 26471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:31.572483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:33365 (local address 127.21.214.67:41439)
0811 01:44:31.573326 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:31.573372 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:31.575093 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:31.575918 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:31.575926 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:31.576473 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:31.577058 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:31.577076 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:31.578632 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:31.578636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:31.580446 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:31.580460 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:31.581468 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:31.581476 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:31.581773 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:31.582521 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:31.582546 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:31.584941 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:31.587908 (+ 2967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:31.587918 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:31.587938 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:31.588397 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:31.588787 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:31.588794 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:31.588798 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:31.588997 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:31.589602 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:31.589611 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:31.589993 (+ 382us) client_negotiation.cc:769] Sending connection context
0811 01:44:31.590294 (+ 301us) client_negotiation.cc:240] Negotiation successful
0811 01:44:31.590886 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":327,"threads_started":1}
I20260811 01:44:31.594025 26468 heartbeater.cc:347] Connected to a master server at 127.21.214.126:33365
I20260811 01:44:31.594446 26468 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:31.595324 26468 heartbeater.cc:511] Master 127.21.214.126:33365 requested a full tablet report, sending...
I20260811 01:44:31.596935 25918 ts_manager.cc:194] Registered new tserver with Master: 63e53e93f2504952bbd6da369929fd61 (127.21.214.67:40653)
I20260811 01:44:31.597872 25918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:41439
I20260811 01:44:31.607343 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 01:44:31 dist-test-slave-9gcw krb5kdc[25862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412671, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 01:44:31 dist-test-slave-9gcw krb5kdc[25862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412671, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:31.672561 26470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:31.656300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43526 (local address 127.21.214.126:33365)
0811 01:44:31.656568 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:31.656578 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:31.656935 (+ 357us) server_negotiation.cc:407] Connection header received
0811 01:44:31.657104 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:31.657108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:31.657385 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:31.657491 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:31.659003 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:31.659849 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:31.661178 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:31.661944 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:31.662078 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:31.666090 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:31.666095 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:31.666100 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:31.666147 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:31.669808 (+ 3661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:31.670513 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:31.670517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:31.670520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:31.670693 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:31.671123 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:31.671126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:31.671129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:31.671539 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:31.671753 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:31.672083 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 01:44:31.672297 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 01:44:31.673405 25918 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43526
I20260811 01:44:31.675186 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 25989
Aug 11 01:44:31 dist-test-slave-9gcw krb5kdc[25862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412671, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:44:31.693765 26482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:31.676472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42980 (local address 127.21.214.65:46295)
0811 01:44:31.677744 (+ 1272us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:31.677751 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:31.677780 (+ 29us) server_negotiation.cc:407] Connection header received
0811 01:44:31.677847 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:31.677853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:31.678203 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:31.678410 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:31.679660 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:31.680692 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:31.681993 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:31.682587 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:31.682733 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:31.687552 (+ 4819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:31.687566 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:31.687583 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:31.687634 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:31.690606 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:31.691376 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:31.691384 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:31.691386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:31.691522 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:31.691971 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:31.691976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:31.691979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:31.692382 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:31.692709 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:31.693070 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 01:44:31.693397 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1103,"thread_start_us":910,"threads_started":1}
W20260811 01:44:31.695255 26129 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42980
W20260811 01:44:31.696626 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 25989: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:44:31.696914 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 25989
I20260811 01:44:31.719034 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 26143
Aug 11 01:44:31 dist-test-slave-9gcw krb5kdc[25862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412671, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.214.66@KRBTEST.COM
I20260811 01:44:31.738123 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:31.720240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53726 (local address 127.21.214.66:37263)
0811 01:44:31.720886 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:31.720894 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:31.720923 (+ 29us) server_negotiation.cc:407] Connection header received
0811 01:44:31.720990 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:31.720998 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:31.721352 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:31.721507 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:31.722857 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:31.724035 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:31.725580 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:31.726195 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:31.726369 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:31.731011 (+ 4642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:31.731025 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:31.731043 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:31.731080 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:31.734836 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:31.735669 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:31.735677 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:31.735680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:31.735832 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:31.736251 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:31.736256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:31.736258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:31.736674 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:31.737040 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:31.737504 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 01:44:31.737751 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":322,"threads_started":1}
W20260811 01:44:31.739689 26295 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53726
W20260811 01:44:31.741029 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 26143: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:44:31.741303 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 26143
I20260811 01:44:31.763784 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 26308
Aug 11 01:44:31 dist-test-slave-9gcw krb5kdc[25862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412671, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.214.67@KRBTEST.COM
I20260811 01:44:31.782318 26484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:31.764825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42846 (local address 127.21.214.67:40653)
0811 01:44:31.765532 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:31.765539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:31.765570 (+ 31us) server_negotiation.cc:407] Connection header received
0811 01:44:31.765635 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:31.765641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:31.765877 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:31.766079 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:31.767559 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:31.768647 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:31.770012 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:31.770610 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:31.770768 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:31.775539 (+ 4771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:31.775560 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:31.775579 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:31.775621 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:31.778972 (+ 3351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:31.779733 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:31.779741 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:31.779744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:31.779885 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:31.780272 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:31.780276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:31.780278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:31.780670 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:31.781015 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:31.781418 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 01:44:31.781856 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":387,"threads_started":1}
W20260811 01:44:31.784327 26460 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42846
W20260811 01:44:31.785722 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 26308: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:44:31.785930 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 26308
I20260811 01:44:31.818049 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 25878
I20260811 01:44:31.819331 25953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:31.926937 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 25878
2026-08-11T01:44:31Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4122 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:44:32 dist-test-slave-9gcw krb5kdc[26492](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:44:32 dist-test-slave-9gcw krb5kdc[26492](info): set up 2 sockets
Aug 11 01:44:32 dist-test-slave-9gcw krb5kdc[26492](info): commencing operation
krb5kdc: starting...
W20260811 01:44:33.137490 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:44:33 dist-test-slave-9gcw krb5kdc[26492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:44:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:44:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:44:33.390740 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:37871
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:44861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:37871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:33.748903 26510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:33.749305 26510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:33.749408 26510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:33.756821 26510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:44:33.756946 26510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:33.756994 26510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:33.757045 26510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:44:33.757090 26510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:44:33.767247 26510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:44861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:37871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:37871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:33.768638 26510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:33.769990 26510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:33.783735 26515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:33.783807 26516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:33.783949 26518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:33.785440 26510 server_base.cc:1034] running on GCE node
I20260811 01:44:33.786484 26510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:33.788221 26510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:33.789492 26510 hybrid_clock.cc:648] HybridClock initialized: now 1786412673789402 us; error 56 us; skew 500 ppm
I20260811 01:44:33.789963 26510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:33 dist-test-slave-9gcw krb5kdc[26492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412673, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:33.794463 26510 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:44:33.796128 26510 webserver.cc:460] Webserver started at http://127.21.214.126:45209/ using document root <none> and password file <none>
I20260811 01:44:33.796898 26510 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:33.797055 26510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:33.797439 26510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:33.800729 26510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "e7d8a75db4964109a6c68f4d8e139be3"
format_stamp: "Formatted at 2026-08-11 01:44:33 on dist-test-slave-9gcw"
server_key: "f9332abe94852d8ff96c2fc370bdb76e"
server_key_iv: "97339f7bfd165bbc25a1a8d3a7f42253"
server_key_version: "encryptionkey@0"
I20260811 01:44:33.801618 26510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "e7d8a75db4964109a6c68f4d8e139be3"
format_stamp: "Formatted at 2026-08-11 01:44:33 on dist-test-slave-9gcw"
server_key: "f9332abe94852d8ff96c2fc370bdb76e"
server_key_iv: "97339f7bfd165bbc25a1a8d3a7f42253"
server_key_version: "encryptionkey@0"
I20260811 01:44:33.806999 26510 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 01:44:33.810768 26528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:33.812166 26510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 01:44:33.812381 26510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "e7d8a75db4964109a6c68f4d8e139be3"
format_stamp: "Formatted at 2026-08-11 01:44:33 on dist-test-slave-9gcw"
server_key: "f9332abe94852d8ff96c2fc370bdb76e"
server_key_iv: "97339f7bfd165bbc25a1a8d3a7f42253"
server_key_version: "encryptionkey@0"
I20260811 01:44:33.812624 26510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:33.832608 26510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:33.837821 26510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:33.838153 26510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:33.857682 26510 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:37871
I20260811 01:44:33.857717 26587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:37871 every 8 connection(s)
I20260811 01:44:33.859285 26510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:44:33.863569 26588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:33.866145 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 26510
I20260811 01:44:33.866401 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:44:33.867045 22361 external_mini_cluster.cc:1428] Setting key d3190094beaf07a5d34605e95a979d44
I20260811 01:44:33.878786 26588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3: Bootstrap starting.
I20260811 01:44:33.882472 26588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3: Neither blocks nor log segments found. Creating new log.
Aug 11 01:44:33 dist-test-slave-9gcw krb5kdc[26492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:33.883713 26588 log.cc:824] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:33.887969 26588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3: No bootstrap required, opened a new log
I20260811 01:44:33.894336 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:33.869303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57090 (local address 127.21.214.126:37871)
0811 01:44:33.870253 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:33.870271 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:33.870334 (+ 63us) server_negotiation.cc:407] Connection header received
0811 01:44:33.873060 (+ 2726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:33.873094 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:33.873716 (+ 622us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:33.874237 (+ 521us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:33.876061 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:33.877657 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:33.878975 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:33.879693 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:33.879908 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:33.884166 (+ 4258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:33.884178 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:33.884200 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:33.884255 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:33.888858 (+ 4603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:33.889721 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:33.889731 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:33.889743 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:33.889975 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:33.890383 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:33.890394 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:33.890398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:33.891371 (+ 973us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:33.891750 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:33.892147 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 01:44:33.892670 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":363,"threads_started":1}
I20260811 01:44:33.898937 26588 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d8a75db4964109a6c68f4d8e139be3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 37871 } }
I20260811 01:44:33.899336 26588 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:33.899417 26588 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7d8a75db4964109a6c68f4d8e139be3, State: Initialized, Role: FOLLOWER
I20260811 01:44:33.899947 26588 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d8a75db4964109a6c68f4d8e139be3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 37871 } }
I20260811 01:44:33.900179 26588 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:44:33.900293 26588 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:44:33.900427 26588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:33.903337 26588 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d8a75db4964109a6c68f4d8e139be3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 37871 } }
I20260811 01:44:33.903838 26588 leader_election.cc:302] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7d8a75db4964109a6c68f4d8e139be3; no voters:
I20260811 01:44:33.904452 26588 leader_election.cc:288] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:44:33.904610 26597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:33.905421 26597 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3 [term 1 LEADER]: Becoming Leader. State: Replica: e7d8a75db4964109a6c68f4d8e139be3, State: Running, Role: LEADER
I20260811 01:44:33.905977 26597 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d8a75db4964109a6c68f4d8e139be3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 37871 } }
I20260811 01:44:33.906857 26588 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:44:33.908828 26598 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7d8a75db4964109a6c68f4d8e139be3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d8a75db4964109a6c68f4d8e139be3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 37871 } } }
I20260811 01:44:33.909255 26598 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3 [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:33.909744 26599 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7d8a75db4964109a6c68f4d8e139be3. Latest consensus state: current_term: 1 leader_uuid: "e7d8a75db4964109a6c68f4d8e139be3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d8a75db4964109a6c68f4d8e139be3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 37871 } } }
I20260811 01:44:33.910247 26599 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3 [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:33.912496 26604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:44:33.918321 26604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:44:33.928474 26604 catalog_manager.cc:1347] Generated new cluster ID: 87e25adb474143caab655be56d33bfed
I20260811 01:44:33.928602 26604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:44:33.956156 26604 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:44:33.957780 26604 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:44:33.966529 26604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7d8a75db4964109a6c68f4d8e139be3: Generated new TSK 0
I20260811 01:44:33.967450 26604 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
I20260811 01:44:34.046872 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:37871
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:44861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:34.397303 26621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:34.397608 26621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:34.397713 26621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:34.405103 26621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:34.405246 26621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:34.405362 26621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:44:34.416760 26621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:44861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:37871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:34.418208 26621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:34.419853 26621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:34.433833 26629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:34.433876 26631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:34.433918 26627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:34.434688 26621 server_base.cc:1034] running on GCE node
I20260811 01:44:34.435376 26621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:34.436342 26621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:34.437557 26621 hybrid_clock.cc:648] HybridClock initialized: now 1786412674437485 us; error 60 us; skew 500 ppm
I20260811 01:44:34.437935 26621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:34 dist-test-slave-9gcw krb5kdc[26492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412674, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:34.441876 26621 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:44:34.443482 26621 webserver.cc:460] Webserver started at http://127.21.214.65:45205/ using document root <none> and password file <none>
I20260811 01:44:34.444227 26621 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:34.444376 26621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:34.444718 26621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:34.447983 26621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "b5aabed741b7451097c178b2853b998c"
format_stamp: "Formatted at 2026-08-11 01:44:34 on dist-test-slave-9gcw"
server_key: "ae996f5827d102ace6f9dd39ca4a8f54"
server_key_iv: "1014546499f340b9ee53f26bebbf923d"
server_key_version: "encryptionkey@0"
I20260811 01:44:34.448835 26621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "b5aabed741b7451097c178b2853b998c"
format_stamp: "Formatted at 2026-08-11 01:44:34 on dist-test-slave-9gcw"
server_key: "ae996f5827d102ace6f9dd39ca4a8f54"
server_key_iv: "1014546499f340b9ee53f26bebbf923d"
server_key_version: "encryptionkey@0"
I20260811 01:44:34.454246 26621 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 01:44:34.457885 26639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:34.459240 26621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:44:34.459426 26621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "b5aabed741b7451097c178b2853b998c"
format_stamp: "Formatted at 2026-08-11 01:44:34 on dist-test-slave-9gcw"
server_key: "ae996f5827d102ace6f9dd39ca4a8f54"
server_key_iv: "1014546499f340b9ee53f26bebbf923d"
server_key_version: "encryptionkey@0"
I20260811 01:44:34.459649 26621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:34.478222 26621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:34.483565 26621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:34.483963 26621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:34.485292 26621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:34.487027 26621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:34.487159 26621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:34.487356 26621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:34.487473 26621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:34.526687 26621 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:33637
I20260811 01:44:34.526724 26779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:33637 every 8 connection(s)
I20260811 01:44:34.528199 26621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:44:34.532299 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 26621
I20260811 01:44:34.532591 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:44:34.533479 22361 external_mini_cluster.cc:1428] Setting key 84b345720dfb2886ccd3f713e060a57e
Aug 11 01:44:34 dist-test-slave-9gcw krb5kdc[26492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412674, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:34.549217 26782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:34.531083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:37311 (local address 127.21.214.126:37871)
0811 01:44:34.531793 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:34.531802 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:34.533224 (+ 1422us) server_negotiation.cc:407] Connection header received
0811 01:44:34.534896 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:34.534902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:34.534994 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:34.535115 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:34.536722 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:34.537532 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:34.539138 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:34.539539 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:34.539679 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:34.543247 (+ 3568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:34.543252 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:34.543256 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:34.543288 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:34.545989 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:34.546745 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:34.546753 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:34.546759 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:34.546951 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:34.547404 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:34.547411 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:34.547414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:34.547684 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:34.547853 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:34.548576 (+ 723us) server_negotiation.cc:299] Negotiation successful
0811 01:44:34.548820 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":320,"threads_started":1}
I20260811 01:44:34.550220 26783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:34.531665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:37871 (local address 127.21.214.65:37311)
0811 01:44:34.532560 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:34.532616 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:34.534613 (+ 1997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:34.535407 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:34.535419 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:34.535966 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:34.536545 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:34.536563 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:34.537709 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:34.537713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:34.538986 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:34.538998 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:34.539799 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:34.539808 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:34.540044 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:34.540752 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:34.540782 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:34.543017 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:34.546141 (+ 3124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:34.546150 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:34.546167 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:34.546598 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:34.547067 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:34.547072 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:34.547076 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:34.547286 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:34.547825 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:34.547833 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:34.548193 (+ 360us) client_negotiation.cc:769] Sending connection context
0811 01:44:34.548485 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 01:44:34.549024 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":351,"threads_started":1}
I20260811 01:44:34.551988 26780 heartbeater.cc:347] Connected to a master server at 127.21.214.126:37871
I20260811 01:44:34.552392 26780 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:34.553532 26780 heartbeater.cc:511] Master 127.21.214.126:37871 requested a full tablet report, sending...
I20260811 01:44:34.556510 26547 ts_manager.cc:194] Registered new tserver with Master: b5aabed741b7451097c178b2853b998c (127.21.214.65:33637)
WARNING: no policy specified for kudu/127.21.214.66@KRBTEST.COM; defaulting to no policy
I20260811 01:44:34.558784 26547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:37311
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
I20260811 01:44:34.590875 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:37871
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:44861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:34.939106 26788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:34.939419 26788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:34.939518 26788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:34.946946 26788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:34.947080 26788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:34.947197 26788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:44:34.957324 26788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:44861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:37871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:34.958623 26788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:34.959944 26788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:34.974483 26795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:34.974524 26796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:34.974562 26798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:34.975433 26788 server_base.cc:1034] running on GCE node
I20260811 01:44:34.976052 26788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:34.977082 26788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:34.978333 26788 hybrid_clock.cc:648] HybridClock initialized: now 1786412674978291 us; error 58 us; skew 500 ppm
I20260811 01:44:34.978684 26788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:34 dist-test-slave-9gcw krb5kdc[26492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412674, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:34.986002 26788 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:44:34.987895 26788 webserver.cc:460] Webserver started at http://127.21.214.66:37479/ using document root <none> and password file <none>
I20260811 01:44:34.988660 26788 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:34.988799 26788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:34.989215 26788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:34.992321 26788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "2da29b89524b4102b829d3b6795db803"
format_stamp: "Formatted at 2026-08-11 01:44:34 on dist-test-slave-9gcw"
server_key: "ba9e972f3f1f06c5c2fdb7bacfa55b1a"
server_key_iv: "27a31c12d035ea86c3f52dece5e18e62"
server_key_version: "encryptionkey@0"
I20260811 01:44:34.993224 26788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "2da29b89524b4102b829d3b6795db803"
format_stamp: "Formatted at 2026-08-11 01:44:34 on dist-test-slave-9gcw"
server_key: "ba9e972f3f1f06c5c2fdb7bacfa55b1a"
server_key_iv: "27a31c12d035ea86c3f52dece5e18e62"
server_key_version: "encryptionkey@0"
I20260811 01:44:34.998840 26788 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 01:44:35.003135 26808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:35.004596 26788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 01:44:35.004877 26788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "2da29b89524b4102b829d3b6795db803"
format_stamp: "Formatted at 2026-08-11 01:44:34 on dist-test-slave-9gcw"
server_key: "ba9e972f3f1f06c5c2fdb7bacfa55b1a"
server_key_iv: "27a31c12d035ea86c3f52dece5e18e62"
server_key_version: "encryptionkey@0"
I20260811 01:44:35.005120 26788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:35.035218 26788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:35.040477 26788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:35.040848 26788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:35.042014 26788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:35.043582 26788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:35.043690 26788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:35.043844 26788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:35.043919 26788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:35.081943 26788 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:36251
I20260811 01:44:35.081987 26939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:36251 every 8 connection(s)
I20260811 01:44:35.083424 26788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:44:35.084728 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 26788
I20260811 01:44:35.084959 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:44:35.085551 22361 external_mini_cluster.cc:1428] Setting key 90b4bd0515352cefe8d79d90e58f7130
Aug 11 01:44:35 dist-test-slave-9gcw krb5kdc[26492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412674, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:35.108709 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:35.086573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:58081 (local address 127.21.214.126:37871)
0811 01:44:35.087081 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:35.087086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:35.088575 (+ 1489us) server_negotiation.cc:407] Connection header received
0811 01:44:35.090777 (+ 2202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:35.090786 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:35.090882 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:35.091014 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:35.092691 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:35.093904 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:35.095665 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:35.096425 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:35.096591 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:35.101046 (+ 4455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:35.101056 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:35.101064 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:35.101111 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:35.105124 (+ 4013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:35.106057 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:35.106064 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:35.106068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:35.106233 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:35.106767 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:35.106776 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:35.106780 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:35.107148 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:35.107309 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:35.108044 (+ 735us) server_negotiation.cc:299] Negotiation successful
0811 01:44:35.108299 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":285,"threads_started":1}
I20260811 01:44:35.109927 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:35.087382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:37871 (local address 127.21.214.66:58081)
0811 01:44:35.088369 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:35.088428 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:35.090466 (+ 2038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:35.091274 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:35.091283 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:35.091851 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:35.092493 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:35.092512 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:35.094092 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:35.094096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:35.095499 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:35.095518 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:35.096728 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:35.096735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:35.096977 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:35.097978 (+ 1001us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:35.098004 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:35.100749 (+ 2745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:35.105364 (+ 4615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:35.105374 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:35.105390 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:35.105902 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:35.106366 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:35.106373 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:35.106381 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:35.106584 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:35.107285 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:35.107294 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:35.107672 (+ 378us) client_negotiation.cc:769] Sending connection context
0811 01:44:35.107966 (+ 294us) client_negotiation.cc:240] Negotiation successful
0811 01:44:35.108581 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":424,"threads_started":1}
WARNING: no policy specified for kudu/127.21.214.67@KRBTEST.COM; defaulting to no policy
I20260811 01:44:35.111625 26940 heartbeater.cc:347] Connected to a master server at 127.21.214.126:37871
I20260811 01:44:35.111977 26940 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:35.112854 26940 heartbeater.cc:511] Master 127.21.214.126:37871 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.67@KRBTEST.COM" created.
I20260811 01:44:35.114607 26547 ts_manager.cc:194] Registered new tserver with Master: 2da29b89524b4102b829d3b6795db803 (127.21.214.66:36251)
I20260811 01:44:35.115842 26547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:58081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
I20260811 01:44:35.145809 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:37871
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:44861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:35.496697 26946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:35.497021 26946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:35.497119 26946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:35.505018 26946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:35.505221 26946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:35.505467 26946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:44:35.516337 26946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:44861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:37871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:35.517848 26946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:35.519249 26946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:35.534466 26957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:35.534502 26954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:35.535100 26953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:35.535454 26946 server_base.cc:1034] running on GCE node
I20260811 01:44:35.536080 26946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:35.537101 26946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:35.538404 26946 hybrid_clock.cc:648] HybridClock initialized: now 1786412675538355 us; error 43 us; skew 500 ppm
I20260811 01:44:35.538803 26946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:35 dist-test-slave-9gcw krb5kdc[26492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412675, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:35.542500 26946 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:44:35.544368 26946 webserver.cc:460] Webserver started at http://127.21.214.67:42847/ using document root <none> and password file <none>
I20260811 01:44:35.545284 26946 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:35.545451 26946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:35.545818 26946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:35.549717 26946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "c751fb34029740fc9ec5baa9e203079c"
format_stamp: "Formatted at 2026-08-11 01:44:35 on dist-test-slave-9gcw"
server_key: "f8a95122fdae53cb3e72dd24cd468879"
server_key_iv: "3f5fbd2d69e0dfa50b481284f74a821c"
server_key_version: "encryptionkey@0"
I20260811 01:44:35.550559 26946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "c751fb34029740fc9ec5baa9e203079c"
format_stamp: "Formatted at 2026-08-11 01:44:35 on dist-test-slave-9gcw"
server_key: "f8a95122fdae53cb3e72dd24cd468879"
server_key_iv: "3f5fbd2d69e0dfa50b481284f74a821c"
server_key_version: "encryptionkey@0"
I20260811 01:44:35.556238 26946 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 01:44:35.560305 26966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:35.561980 26946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:44:35.562201 26946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "c751fb34029740fc9ec5baa9e203079c"
format_stamp: "Formatted at 2026-08-11 01:44:35 on dist-test-slave-9gcw"
server_key: "f8a95122fdae53cb3e72dd24cd468879"
server_key_iv: "3f5fbd2d69e0dfa50b481284f74a821c"
server_key_version: "encryptionkey@0"
I20260811 01:44:35.562568 26780 heartbeater.cc:503] Master 127.21.214.126:37871 was elected leader, sending a full tablet report...
I20260811 01:44:35.562481 26946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:35.592810 26946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:35.597920 26946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:35.598310 26946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:35.599509 26946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:35.601231 26946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:35.601357 26946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 01:44:35.601545 26946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:35.601644 26946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:35.641923 26946 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:39401
I20260811 01:44:35.641969 27095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:39401 every 8 connection(s)
I20260811 01:44:35.643539 26946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:44:35.651520 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 26946
I20260811 01:44:35.651695 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:44:35.652168 22361 external_mini_cluster.cc:1428] Setting key d2837b08d78479e11458f70ee76ca253
Aug 11 01:44:35 dist-test-slave-9gcw krb5kdc[26492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412675, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:35.667644 27098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:35.646503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:52473 (local address 127.21.214.126:37871)
0811 01:44:35.647165 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:35.647171 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:35.648060 (+ 889us) server_negotiation.cc:407] Connection header received
0811 01:44:35.650418 (+ 2358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:35.650425 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:35.650550 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:35.650702 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:35.652750 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:35.653791 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:35.655592 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:35.656069 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:35.656220 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:35.660222 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:35.660229 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:35.660234 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:35.660281 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:35.663318 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:35.664370 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:35.664378 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:35.664384 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:35.664578 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:35.665136 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:35.665171 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:35.665176 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:35.665518 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:35.665750 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:35.666674 (+ 924us) server_negotiation.cc:299] Negotiation successful
0811 01:44:35.666942 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":386,"threads_started":1}
I20260811 01:44:35.668723 27099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:35.646957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:37871 (local address 127.21.214.67:52473)
0811 01:44:35.647761 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:35.647833 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:35.650084 (+ 2251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:35.651016 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:35.651026 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:35.651785 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:35.652539 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:35.652562 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:35.653984 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:35.653989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:35.655418 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:35.655431 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:35.656341 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:35.656350 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:35.656614 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:35.657390 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:35.657419 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:35.659952 (+ 2533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:35.663562 (+ 3610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:35.663575 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:35.663603 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:35.664205 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:35.664734 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:35.664741 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:35.664752 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:35.664984 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:35.665700 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:35.665711 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:35.666191 (+ 480us) client_negotiation.cc:769] Sending connection context
0811 01:44:35.666546 (+ 355us) client_negotiation.cc:240] Negotiation successful
0811 01:44:35.667212 (+ 666us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":314,"threads_started":1}
I20260811 01:44:35.670331 27096 heartbeater.cc:347] Connected to a master server at 127.21.214.126:37871
I20260811 01:44:35.670629 27096 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:35.671392 27096 heartbeater.cc:511] Master 127.21.214.126:37871 requested a full tablet report, sending...
I20260811 01:44:35.672973 26547 ts_manager.cc:194] Registered new tserver with Master: c751fb34029740fc9ec5baa9e203079c (127.21.214.67:39401)
I20260811 01:44:35.674003 26547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:52473
I20260811 01:44:35.680415 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 01:44:35 dist-test-slave-9gcw krb5kdc[26492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 01:44:35 dist-test-slave-9gcw krb5kdc[26492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:44:35.740379 27105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:35.724099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54492 (local address 127.21.214.65:33637)
0811 01:44:35.724914 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:35.724922 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:35.724947 (+ 25us) server_negotiation.cc:407] Connection header received
0811 01:44:35.725011 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:35.725017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:35.725315 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:35.725491 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:35.726989 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:35.728044 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:35.729405 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:35.730007 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:35.730160 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:35.734364 (+ 4204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:35.734376 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:35.734393 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:35.734428 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:35.737348 (+ 2920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:35.738063 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:35.738071 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:35.738073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:35.738206 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:35.738639 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:35.738645 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:35.738648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:35.739032 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:35.739369 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:35.739706 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 01:44:35.740005 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":338,"threads_started":1}
Aug 11 01:44:35 dist-test-slave-9gcw krb5kdc[26492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:35.759544 27098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:35.745283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57092 (local address 127.21.214.126:37871)
0811 01:44:35.745525 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:35.745537 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:35.745652 (+ 115us) server_negotiation.cc:407] Connection header received
0811 01:44:35.745787 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:35.745791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:35.745880 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:35.745973 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:35.747240 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:35.748065 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:35.749408 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:35.749869 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:35.749978 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:35.754491 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:35.754499 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:35.754505 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:35.754552 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:35.757099 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:35.757828 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:35.757832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:35.757835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:35.757959 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:35.758283 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:35.758287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:35.758289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:35.758603 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:35.758795 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:35.759037 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 01:44:35.759217 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 01:44:35.760193 26547 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57092
I20260811 01:44:35.761337 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 26621
I20260811 01:44:35.774688 27105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:35.762222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54508 (local address 127.21.214.65:33637)
0811 01:44:35.762397 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:35.762401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:35.762684 (+ 283us) server_negotiation.cc:407] Connection header received
0811 01:44:35.762942 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:35.762946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:35.763010 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:35.763094 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:35.764333 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:35.765121 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:35.766368 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:35.766780 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:35.766939 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:35.769299 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:35.769304 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:35.769307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:35.769340 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:35.772147 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:35.772749 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:35.772753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:35.772755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:35.772888 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:35.773575 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:35.773578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:35.773581 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:35.773834 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:35.774018 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:35.774286 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 01:44:35.774454 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 01:44:35.775362 26775 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:35.862854 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 26621
I20260811 01:44:35.885979 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 26788
Aug 11 01:44:35 dist-test-slave-9gcw krb5kdc[26492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.66@KRBTEST.COM
I20260811 01:44:35.903162 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:35.887129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54936 (local address 127.21.214.66:36251)
0811 01:44:35.887719 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:35.887726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:35.887753 (+ 27us) server_negotiation.cc:407] Connection header received
0811 01:44:35.887818 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:35.887825 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:35.888048 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:35.888212 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:35.889669 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:35.890871 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:35.892187 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:35.892758 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:35.892866 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:35.897316 (+ 4450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:35.897331 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:35.897347 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:35.897394 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:35.900331 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:35.900994 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:35.901001 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:35.901004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:35.901330 (+ 326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:35.901672 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:35.901675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:35.901677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:35.902041 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:35.902274 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:35.902557 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 01:44:35.902839 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":317,"threads_started":1}
I20260811 01:44:35.904484 26934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:35.990942 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 26788
I20260811 01:44:36.013047 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 26946
Aug 11 01:44:36 dist-test-slave-9gcw krb5kdc[26492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.67@KRBTEST.COM
I20260811 01:44:36.031843 27114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:36.014197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36568 (local address 127.21.214.67:39401)
0811 01:44:36.014968 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:36.014975 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:36.015007 (+ 32us) server_negotiation.cc:407] Connection header received
0811 01:44:36.015072 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:36.015079 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:36.015309 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:36.015459 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:36.016816 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:36.018071 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:36.019374 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:36.019948 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:36.020058 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:36.025510 (+ 5452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:36.025526 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:36.025543 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:36.025658 (+ 115us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:36.028994 (+ 3336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:36.029740 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:36.029748 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:36.029751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:36.029907 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:36.030263 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:36.030267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:36.030269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:36.030778 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:36.031074 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:36.031265 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 01:44:36.031497 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":359,"threads_started":1}
I20260811 01:44:36.033406 27091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:36.117643 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 26946
I20260811 01:44:36.137916 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 26510
I20260811 01:44:36.139400 26583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:36.245733 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 26510
2026-08-11T01:44:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4316 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:44:36 dist-test-slave-9gcw krb5kdc[27121](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:44:36 dist-test-slave-9gcw krb5kdc[27121](info): set up 2 sockets
Aug 11 01:44:36 dist-test-slave-9gcw krb5kdc[27121](info): commencing operation
krb5kdc: starting...
W20260811 01:44:37.454047 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.005s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:44:37 dist-test-slave-9gcw krb5kdc[27121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:44:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:44:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:44:37.706393 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:45319
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:45319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:38.063908 27135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:38.064237 27135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:38.064388 27135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:38.071988 27135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:44:38.072110 27135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:38.072165 27135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:38.072263 27135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:44:38.072363 27135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:44:38.083175 27135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:45319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:45319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:38.084614 27135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:38.086140 27135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:38.099363 27141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:38.099386 27144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:38.099303 27142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:38.100407 27135 server_base.cc:1034] running on GCE node
I20260811 01:44:38.101351 27135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:38.102828 27135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:38.104043 27135 hybrid_clock.cc:648] HybridClock initialized: now 1786412678103999 us; error 33 us; skew 500 ppm
I20260811 01:44:38.104445 27135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:38 dist-test-slave-9gcw krb5kdc[27121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412678, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:38.107973 27135 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:44:38.109375 27135 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:38.109505 27135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:38.109872 27135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:38.112977 27135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "c1663b7097a04c7babecf998bf825985"
format_stamp: "Formatted at 2026-08-11 01:44:38 on dist-test-slave-9gcw"
server_key: "e184e898155d5139e06015b206bd600e"
server_key_iv: "c2fa89b3bb3e5a2e60e9ae3ea14341b6"
server_key_version: "encryptionkey@0"
I20260811 01:44:38.113929 27135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "c1663b7097a04c7babecf998bf825985"
format_stamp: "Formatted at 2026-08-11 01:44:38 on dist-test-slave-9gcw"
server_key: "e184e898155d5139e06015b206bd600e"
server_key_iv: "c2fa89b3bb3e5a2e60e9ae3ea14341b6"
server_key_version: "encryptionkey@0"
I20260811 01:44:38.119465 27135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 01:44:38.123732 27151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:38.125509 27135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:44:38.125761 27135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "c1663b7097a04c7babecf998bf825985"
format_stamp: "Formatted at 2026-08-11 01:44:38 on dist-test-slave-9gcw"
server_key: "e184e898155d5139e06015b206bd600e"
server_key_iv: "c2fa89b3bb3e5a2e60e9ae3ea14341b6"
server_key_version: "encryptionkey@0"
I20260811 01:44:38.126048 27135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:38.142623 27135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:38.147884 27135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:38.148231 27135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:38.169445 27135 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:45319
I20260811 01:44:38.169467 27211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:45319 every 8 connection(s)
I20260811 01:44:38.170552 27135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:44:38.172647 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 27135
I20260811 01:44:38.172905 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:44:38.173794 22361 external_mini_cluster.cc:1428] Setting key cbaec2b23f777b13ca4a3f982c974a24
I20260811 01:44:38.175750 27212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 01:44:38 dist-test-slave-9gcw krb5kdc[27121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:38.192884 27212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985: Bootstrap starting.
I20260811 01:44:38.198103 27212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:38.199561 27212 log.cc:824] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:38.203570 27212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985: No bootstrap required, opened a new log
I20260811 01:44:38.203893 27215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:38.176285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48408 (local address 127.21.214.126:45319)
0811 01:44:38.177414 (+ 1129us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:38.177429 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:38.177476 (+ 47us) server_negotiation.cc:407] Connection header received
0811 01:44:38.179396 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:38.179422 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:38.179887 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:38.180304 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:38.182681 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:38.184301 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:38.186367 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:38.187261 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:38.187416 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:38.193189 (+ 5773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:38.193204 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:38.193228 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:38.193314 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:38.198101 (+ 4787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:38.199066 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:38.199076 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:38.199086 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:38.199359 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:38.199856 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:38.199861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:38.199864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:38.200716 (+ 852us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:38.200972 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:38.201852 (+ 880us) server_negotiation.cc:299] Negotiation successful
0811 01:44:38.202351 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":663,"thread_start_us":418,"threads_started":1}
I20260811 01:44:38.212781 27212 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1663b7097a04c7babecf998bf825985" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45319 } }
I20260811 01:44:38.213275 27212 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:38.213421 27212 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1663b7097a04c7babecf998bf825985, State: Initialized, Role: FOLLOWER
I20260811 01:44:38.214195 27212 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1663b7097a04c7babecf998bf825985" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45319 } }
I20260811 01:44:38.214437 27212 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:44:38.214550 27212 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:44:38.214696 27212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:38.217105 27212 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1663b7097a04c7babecf998bf825985" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45319 } }
I20260811 01:44:38.217702 27212 leader_election.cc:302] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1663b7097a04c7babecf998bf825985; no voters:
I20260811 01:44:38.218299 27212 leader_election.cc:288] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:44:38.218513 27217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:38.219293 27217 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985 [term 1 LEADER]: Becoming Leader. State: Replica: c1663b7097a04c7babecf998bf825985, State: Running, Role: LEADER
I20260811 01:44:38.219910 27217 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1663b7097a04c7babecf998bf825985" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45319 } }
I20260811 01:44:38.220940 27212 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:44:38.223249 27218 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1663b7097a04c7babecf998bf825985" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1663b7097a04c7babecf998bf825985" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45319 } } }
I20260811 01:44:38.223308 27219 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1663b7097a04c7babecf998bf825985. Latest consensus state: current_term: 1 leader_uuid: "c1663b7097a04c7babecf998bf825985" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1663b7097a04c7babecf998bf825985" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45319 } } }
I20260811 01:44:38.223723 27218 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985 [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:38.223731 27219 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985 [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:38.227615 27226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:44:38.234220 27226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:44:38.245265 27226 catalog_manager.cc:1347] Generated new cluster ID: 59b89b50973b4a2f99bf6774a6a3126e
I20260811 01:44:38.245414 27226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:44:38.280658 27226 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:44:38.282228 27226 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:44:38.289165 27226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985: Generated new TSK 0
I20260811 01:44:38.290185 27226 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
I20260811 01:44:38.374424 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:45319
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:38.742015 27242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:38.742338 27242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:38.742501 27242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:38.749843 27242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:38.749969 27242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:38.750167 27242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:44:38.760344 27242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:45319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:38.761804 27242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:38.763250 27242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:38.776889 27250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:38.777040 27249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:38.777066 27253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:38.777667 27242 server_base.cc:1034] running on GCE node
I20260811 01:44:38.778530 27242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:38.779603 27242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:38.780867 27242 hybrid_clock.cc:648] HybridClock initialized: now 1786412678780817 us; error 52 us; skew 500 ppm
I20260811 01:44:38.781291 27242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:38 dist-test-slave-9gcw krb5kdc[27121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412678, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:38.784647 27242 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:44:38.785991 27242 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:38.786136 27242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:38.786480 27242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:38.790056 27242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "bcde43dd84354696a1594b38e4dd9bd6"
format_stamp: "Formatted at 2026-08-11 01:44:38 on dist-test-slave-9gcw"
server_key: "910f944909f667dd076469550b54bbe0"
server_key_iv: "66bc2dcc0504c8d64e43061308c2f871"
server_key_version: "encryptionkey@0"
I20260811 01:44:38.790966 27242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "bcde43dd84354696a1594b38e4dd9bd6"
format_stamp: "Formatted at 2026-08-11 01:44:38 on dist-test-slave-9gcw"
server_key: "910f944909f667dd076469550b54bbe0"
server_key_iv: "66bc2dcc0504c8d64e43061308c2f871"
server_key_version: "encryptionkey@0"
I20260811 01:44:38.796712 27242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 01:44:38.800873 27259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:38.802311 27242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 01:44:38.802505 27242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "bcde43dd84354696a1594b38e4dd9bd6"
format_stamp: "Formatted at 2026-08-11 01:44:38 on dist-test-slave-9gcw"
server_key: "910f944909f667dd076469550b54bbe0"
server_key_iv: "66bc2dcc0504c8d64e43061308c2f871"
server_key_version: "encryptionkey@0"
I20260811 01:44:38.802739 27242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:38.821727 27242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:38.826222 27242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:38.826537 27242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:38.827698 27242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:38.829332 27242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:38.829456 27242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:38.829558 27242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:38.829620 27242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:38.864127 27242 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:38577
I20260811 01:44:38.864223 27386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:38577 every 8 connection(s)
I20260811 01:44:38.865093 27242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:44:38.875384 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 27242
I20260811 01:44:38.875636 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:44:38.876367 22361 external_mini_cluster.cc:1428] Setting key bb25be6323dc4df72d4e437f217e91ca
Aug 11 01:44:38 dist-test-slave-9gcw krb5kdc[27121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412678, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:38.886113 27389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:38.867941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:55879 (local address 127.21.214.126:45319)
0811 01:44:38.868404 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:38.868411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:38.869538 (+ 1127us) server_negotiation.cc:407] Connection header received
0811 01:44:38.871371 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:38.871376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:38.871444 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:38.871545 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:38.873046 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:38.873850 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:38.875855 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:38.876243 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:38.876391 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:38.880177 (+ 3786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:38.880181 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:38.880185 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:38.880218 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:38.882889 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:38.883656 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:38.883662 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:38.883672 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:38.883794 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:38.884236 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:38.884242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:38.884245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:38.884535 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:38.884691 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:38.885496 (+ 805us) server_negotiation.cc:299] Negotiation successful
0811 01:44:38.885728 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":205,"threads_started":1}
I20260811 01:44:38.887161 27390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:38.868568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:45319 (local address 127.21.214.65:55879)
0811 01:44:38.869368 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:38.869419 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:38.871090 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:38.871747 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:38.871755 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:38.872301 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:38.872879 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:38.872901 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:38.874005 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:38.874008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:38.875680 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:38.875699 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:38.876512 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:38.876522 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:38.876798 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:38.877598 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:38.877623 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:38.879941 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:38.883042 (+ 3101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:38.883051 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:38.883070 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:38.883521 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:38.883910 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:38.883915 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:38.883919 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:38.884116 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:38.884687 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:38.884696 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:38.885050 (+ 354us) client_negotiation.cc:769] Sending connection context
0811 01:44:38.885398 (+ 348us) client_negotiation.cc:240] Negotiation successful
0811 01:44:38.885978 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":302,"threads_started":1}
I20260811 01:44:38.888896 27387 heartbeater.cc:347] Connected to a master server at 127.21.214.126:45319
I20260811 01:44:38.889274 27387 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:38.890151 27387 heartbeater.cc:511] Master 127.21.214.126:45319 requested a full tablet report, sending...
I20260811 01:44:38.892563 27169 ts_manager.cc:194] Registered new tserver with Master: bcde43dd84354696a1594b38e4dd9bd6 (127.21.214.65:38577)
I20260811 01:44:38.894345 27169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:55879
WARNING: no policy specified for kudu/127.21.214.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
I20260811 01:44:38.938817 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:45319
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:39.292789 27394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:39.293109 27394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:39.293287 27394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:39.300948 27394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:39.301060 27394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:39.301251 27394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:44:39.311780 27394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:45319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:39.313421 27394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:39.314893 27394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:39.329433 27399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:39.329572 27402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:39.329370 27405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:39.331307 27394 server_base.cc:1034] running on GCE node
I20260811 01:44:39.332006 27394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:39.333242 27394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:39.334466 27394 hybrid_clock.cc:648] HybridClock initialized: now 1786412679334403 us; error 62 us; skew 500 ppm
I20260811 01:44:39.334836 27394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:39 dist-test-slave-9gcw krb5kdc[27121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412679, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:39.338383 27394 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:44:39.339723 27394 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:39.339847 27394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:39.340209 27394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:39.343499 27394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "255d773a238d45cf955ef2805d4d1866"
format_stamp: "Formatted at 2026-08-11 01:44:39 on dist-test-slave-9gcw"
server_key: "cf4d10a8243f34561d7a371ac53d3e52"
server_key_iv: "aa69dfbf07f59f8ed1df7054e302cfdc"
server_key_version: "encryptionkey@0"
I20260811 01:44:39.344393 27394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "255d773a238d45cf955ef2805d4d1866"
format_stamp: "Formatted at 2026-08-11 01:44:39 on dist-test-slave-9gcw"
server_key: "cf4d10a8243f34561d7a371ac53d3e52"
server_key_iv: "aa69dfbf07f59f8ed1df7054e302cfdc"
server_key_version: "encryptionkey@0"
I20260811 01:44:39.350406 27394 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260811 01:44:39.354357 27411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:39.355945 27394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 01:44:39.356139 27394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "255d773a238d45cf955ef2805d4d1866"
format_stamp: "Formatted at 2026-08-11 01:44:39 on dist-test-slave-9gcw"
server_key: "cf4d10a8243f34561d7a371ac53d3e52"
server_key_iv: "aa69dfbf07f59f8ed1df7054e302cfdc"
server_key_version: "encryptionkey@0"
I20260811 01:44:39.356389 27394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:39.382217 27394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:39.387023 27394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:39.387357 27394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:39.388404 27394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:39.389943 27394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:39.390075 27394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:39.390219 27394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:39.390311 27394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:39.427006 27394 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:42971
I20260811 01:44:39.427106 27540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:42971 every 8 connection(s)
I20260811 01:44:39.428041 27394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:44:39.434969 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 27394
I20260811 01:44:39.435211 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:44:39.435824 22361 external_mini_cluster.cc:1428] Setting key e5673a820e151e7c37501d30ef171478
Aug 11 01:44:39 dist-test-slave-9gcw krb5kdc[27121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412679, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:39.450703 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:39.431266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:44289 (local address 127.21.214.126:45319)
0811 01:44:39.431921 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:39.431926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:39.432726 (+ 800us) server_negotiation.cc:407] Connection header received
0811 01:44:39.434900 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:39.434904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:39.434977 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:39.435093 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:39.437336 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:39.438097 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:39.439708 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:39.440155 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:39.440313 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:39.444101 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:39.444106 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:39.444110 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:39.444146 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:39.447302 (+ 3156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:39.448187 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:39.448192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:39.448194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:39.448331 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:39.448799 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:39.448810 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:39.448813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:39.449108 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:39.449310 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:39.450060 (+ 750us) server_negotiation.cc:299] Negotiation successful
0811 01:44:39.450314 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":377,"threads_started":1}
I20260811 01:44:39.451807 27544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:39.431728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:45319 (local address 127.21.214.66:44289)
0811 01:44:39.432543 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:39.432582 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:39.434521 (+ 1939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:39.435393 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:39.435405 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:39.436236 (+ 831us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:39.437098 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:39.437121 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:39.438270 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:39.438276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:39.439540 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:39.439552 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:39.440407 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:39.440414 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:39.440665 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:39.441436 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:39.441460 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:39.443844 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:39.447511 (+ 3667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:39.447522 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:39.447541 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:39.448031 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:39.448455 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:39.448460 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:39.448464 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:39.448666 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:39.449282 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:39.449290 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:39.449671 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 01:44:39.449960 (+ 289us) client_negotiation.cc:240] Negotiation successful
0811 01:44:39.450551 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":360,"threads_started":1}
I20260811 01:44:39.453528 27541 heartbeater.cc:347] Connected to a master server at 127.21.214.126:45319
I20260811 01:44:39.453863 27541 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:39.454563 27541 heartbeater.cc:511] Master 127.21.214.126:45319 requested a full tablet report, sending...
I20260811 01:44:39.456378 27169 ts_manager.cc:194] Registered new tserver with Master: 255d773a238d45cf955ef2805d4d1866 (127.21.214.66:42971)
I20260811 01:44:39.457686 27169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:44289
WARNING: no policy specified for kudu/127.21.214.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
I20260811 01:44:39.494268 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:45319
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:39.847503 27547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:39.847800 27547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:39.847903 27547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:39.855298 27547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:39.855443 27547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:39.855563 27547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:44:39.865978 27547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:45319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:39.867350 27547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:39.868813 27547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:39.881976 27552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:39.882138 27556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:39.882143 27554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:39.882985 27547 server_base.cc:1034] running on GCE node
I20260811 01:44:39.883639 27547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:39.884631 27547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:39.885892 27547 hybrid_clock.cc:648] HybridClock initialized: now 1786412679885836 us; error 48 us; skew 500 ppm
I20260811 01:44:39.886250 27547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:39 dist-test-slave-9gcw krb5kdc[27121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412679, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:39.889743 27547 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:44:39.891081 27547 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:39.891222 27547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:39.891575 27547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:39.894800 27547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "3173464f959549c698515078af36b1e1"
format_stamp: "Formatted at 2026-08-11 01:44:39 on dist-test-slave-9gcw"
server_key: "95b4e1bd202363238cd2c6c004e1b391"
server_key_iv: "e370c178d4c116234c494b63272e1cda"
server_key_version: "encryptionkey@0"
I20260811 01:44:39.895700 27547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "3173464f959549c698515078af36b1e1"
format_stamp: "Formatted at 2026-08-11 01:44:39 on dist-test-slave-9gcw"
server_key: "95b4e1bd202363238cd2c6c004e1b391"
server_key_iv: "e370c178d4c116234c494b63272e1cda"
server_key_version: "encryptionkey@0"
I20260811 01:44:39.898794 27387 heartbeater.cc:503] Master 127.21.214.126:45319 was elected leader, sending a full tablet report...
I20260811 01:44:39.902060 27547 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 01:44:39.905973 27564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:39.907759 27547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:44:39.907996 27547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "3173464f959549c698515078af36b1e1"
format_stamp: "Formatted at 2026-08-11 01:44:39 on dist-test-slave-9gcw"
server_key: "95b4e1bd202363238cd2c6c004e1b391"
server_key_iv: "e370c178d4c116234c494b63272e1cda"
server_key_version: "encryptionkey@0"
I20260811 01:44:39.908281 27547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:39.922721 27547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:39.927796 27547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:39.928162 27547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:39.929451 27547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:39.931277 27547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:39.931401 27547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:39.931550 27547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:39.931674 27547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:39.970681 27547 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:32961
I20260811 01:44:39.970829 27691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:32961 every 8 connection(s)
I20260811 01:44:39.971776 27547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:44:39.980455 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 27547
I20260811 01:44:39.980696 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:44:39.981338 22361 external_mini_cluster.cc:1428] Setting key bf9ecb970a094909a6f8ecea2ecb99bb
Aug 11 01:44:39 dist-test-slave-9gcw krb5kdc[27121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412679, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:40.000200 27694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:39.974717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:42799 (local address 127.21.214.126:45319)
0811 01:44:39.975295 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:39.975302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:39.976319 (+ 1017us) server_negotiation.cc:407] Connection header received
0811 01:44:39.978209 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:39.978214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:39.978297 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:39.978404 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:39.980081 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:39.981506 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:39.983514 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:39.984027 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:39.984133 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:39.992791 (+ 8658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:39.992798 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:39.992802 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:39.992845 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:39.996499 (+ 3654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:39.997786 (+ 1287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:39.997792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:39.997795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:39.997954 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:39.998432 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:39.998436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:39.998438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:39.998753 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:39.998873 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:39.999725 (+ 852us) server_negotiation.cc:299] Negotiation successful
0811 01:44:39.999919 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":253,"threads_started":1}
I20260811 01:44:40.001341 27695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:39.975325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:45319 (local address 127.21.214.67:42799)
0811 01:44:39.976133 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:39.976184 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:39.977929 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:39.978714 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:39.978723 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:39.979260 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:39.979843 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:39.979860 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:39.981761 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:39.981765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:39.983345 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:39.983362 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:39.984292 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:39.984301 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:39.984598 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:39.989854 (+ 5256us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:39.989881 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:39.992393 (+ 2512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:39.996704 (+ 4311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:39.996718 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:39.996741 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:39.997634 (+ 893us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:39.998088 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:39.998091 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:39.998094 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:39.998310 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:39.998865 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:39.998871 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:39.999275 (+ 404us) client_negotiation.cc:769] Sending connection context
0811 01:44:39.999542 (+ 267us) client_negotiation.cc:240] Negotiation successful
0811 01:44:39.999981 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":332,"threads_started":1}
I20260811 01:44:40.002844 27692 heartbeater.cc:347] Connected to a master server at 127.21.214.126:45319
I20260811 01:44:40.003196 27692 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:40.003877 27692 heartbeater.cc:511] Master 127.21.214.126:45319 requested a full tablet report, sending...
I20260811 01:44:40.005477 27169 ts_manager.cc:194] Registered new tserver with Master: 3173464f959549c698515078af36b1e1 (127.21.214.67:32961)
I20260811 01:44:40.006331 27169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:42799
I20260811 01:44:40.010387 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:44:40.028062 27694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.015325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48422 (local address 127.21.214.126:45319)
0811 01:44:40.015525 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:40.015530 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:40.016065 (+ 535us) server_negotiation.cc:407] Connection header received
0811 01:44:40.016223 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:40.016227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:40.016291 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:40.016377 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:40.017901 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.018665 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.019896 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.020312 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.020408 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:40.022877 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:40.022882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:40.022885 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:40.022917 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:40.025544 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:40.026136 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:40.026139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:40.026141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:40.026271 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:40.026597 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:40.026600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:40.026602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:40.026917 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:40.027103 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:40.027532 (+ 429us) server_negotiation.cc:299] Negotiation successful
0811 01:44:40.027754 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 01:44:40.032779 27169 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:44:40.039815 27169 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:44:40.068120 27708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.054234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:38577 (local address 127.0.0.1:42862)
0811 01:44:40.056249 (+ 2015us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:40.056328 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:40.056563 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:40.057808 (+ 1245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:40.057817 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:40.057846 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:40.058231 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:40.058252 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.060839 (+ 2587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.060847 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:40.063920 (+ 3073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:40.063944 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.066744 (+ 2800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.066754 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:40.067092 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:40.067170 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 01:44:40.067358 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 01:44:40.067657 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1679,"thread_start_us":1523,"threads_started":1}
I20260811 01:44:40.068596 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.054311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42862 (local address 127.21.214.65:38577)
0811 01:44:40.055777 (+ 1466us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:40.055790 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:40.056799 (+ 1009us) server_negotiation.cc:407] Connection header received
0811 01:44:40.056933 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:40.056945 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:40.057304 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:40.057513 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:40.058468 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.060589 (+ 2121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.064196 (+ 3607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.066078 (+ 1882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.066224 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:40.066573 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:40.068046 (+ 1473us) server_negotiation.cc:299] Negotiation successful
0811 01:44:40.068249 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1241,"thread_start_us":1066,"threads_started":1}
I20260811 01:44:40.073073 27311 tablet_service.cc:1467] Processing CreateTablet for tablet eb0de64da56b4c719e1e077312f45c08 (DEFAULT_TABLE table=test-table [id=1ab3133f203f4e11b5ada7c4c3288203]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:40.074608 27311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb0de64da56b4c719e1e077312f45c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:40.076112 27714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.054805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:32961 (local address 127.0.0.1:39912)
0811 01:44:40.067095 (+ 12290us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:40.067119 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:40.067254 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:40.068935 (+ 1681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:40.068942 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:40.068971 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:40.069589 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:40.069600 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.071619 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.071624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:40.073390 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:40.073403 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.075178 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.075185 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:40.075388 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:40.075422 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 01:44:40.075509 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 01:44:40.075642 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12133,"mutex_wait_us":62,"spinlock_wait_cycles":11008,"thread_start_us":12891,"threads_started":1}
I20260811 01:44:40.077026 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.057585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:42971 (local address 127.0.0.1:53860)
0811 01:44:40.068041 (+ 10456us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:40.068059 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:40.068187 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:40.069009 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:40.069013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:40.069062 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:40.069577 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:40.069592 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.071580 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.071585 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:40.073890 (+ 2305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:40.073907 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.076124 (+ 2217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.076130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:40.076300 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:40.076332 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 01:44:40.076421 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 01:44:40.076557 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10285,"mutex_wait_us":6,"spinlock_wait_cycles":8832,"thread_start_us":9185,"threads_started":1}
I20260811 01:44:40.077724 27711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.055618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39912 (local address 127.21.214.67:32961)
0811 01:44:40.067446 (+ 11828us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:40.067570 (+ 124us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:40.067615 (+ 45us) server_negotiation.cc:407] Connection header received
0811 01:44:40.067735 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:40.067744 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:40.068062 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:40.068636 (+ 574us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:40.069867 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.071487 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.073549 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.074958 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.075096 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:40.075352 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:40.076842 (+ 1490us) server_negotiation.cc:299] Negotiation successful
0811 01:44:40.077033 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11624,"thread_start_us":11681,"threads_started":1}
I20260811 01:44:40.077869 27712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.057232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53860 (local address 127.21.214.66:42971)
0811 01:44:40.066533 (+ 9301us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:40.066546 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:40.068321 (+ 1775us) server_negotiation.cc:407] Connection header received
0811 01:44:40.068424 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:40.068431 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:40.068674 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:40.068874 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:40.069788 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.071412 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.074069 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.075589 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.075747 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:40.075948 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:40.077331 (+ 1383us) server_negotiation.cc:299] Negotiation successful
0811 01:44:40.077526 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9090,"thread_start_us":8939,"threads_started":1}
I20260811 01:44:40.081444 27467 tablet_service.cc:1467] Processing CreateTablet for tablet eb0de64da56b4c719e1e077312f45c08 (DEFAULT_TABLE table=test-table [id=1ab3133f203f4e11b5ada7c4c3288203]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:40.081575 27618 tablet_service.cc:1467] Processing CreateTablet for tablet eb0de64da56b4c719e1e077312f45c08 (DEFAULT_TABLE table=test-table [id=1ab3133f203f4e11b5ada7c4c3288203]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:40.082718 27467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb0de64da56b4c719e1e077312f45c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:40.083056 27618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb0de64da56b4c719e1e077312f45c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:40.090111 27716 tablet_bootstrap.cc:492] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6: Bootstrap starting.
I20260811 01:44:40.094544 27716 tablet_bootstrap.cc:654] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:40.095950 27716 log.cc:824] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:40.096048 27718 tablet_bootstrap.cc:492] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866: Bootstrap starting.
I20260811 01:44:40.097959 27720 tablet_bootstrap.cc:492] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1: Bootstrap starting.
I20260811 01:44:40.099962 27716 tablet_bootstrap.cc:492] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6: No bootstrap required, opened a new log
I20260811 01:44:40.100265 27716 ts_tablet_manager.cc:1397] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260811 01:44:40.100607 27718 tablet_bootstrap.cc:654] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:40.101789 27718 log.cc:824] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:40.101848 27720 tablet_bootstrap.cc:654] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:40.103044 27720 log.cc:824] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:40.105278 27718 tablet_bootstrap.cc:492] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866: No bootstrap required, opened a new log
I20260811 01:44:40.105597 27718 ts_tablet_manager.cc:1397] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 01:44:40.107805 27720 tablet_bootstrap.cc:492] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1: No bootstrap required, opened a new log
I20260811 01:44:40.108072 27720 ts_tablet_manager.cc:1397] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 01:44:40.111594 27716 raft_consensus.cc:348] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcde43dd84354696a1594b38e4dd9bd6" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38577 } } peers { permanent_uuid: "255d773a238d45cf955ef2805d4d1866" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42971 } } peers { permanent_uuid: "3173464f959549c698515078af36b1e1" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 32961 } }
I20260811 01:44:40.112121 27716 raft_consensus.cc:374] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:40.112260 27716 raft_consensus.cc:729] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcde43dd84354696a1594b38e4dd9bd6, State: Initialized, Role: FOLLOWER
I20260811 01:44:40.113076 27716 consensus_queue.cc:260] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcde43dd84354696a1594b38e4dd9bd6" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38577 } } peers { permanent_uuid: "255d773a238d45cf955ef2805d4d1866" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42971 } } peers { permanent_uuid: "3173464f959549c698515078af36b1e1" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 32961 } }
I20260811 01:44:40.113844 27718 raft_consensus.cc:348] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcde43dd84354696a1594b38e4dd9bd6" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38577 } } peers { permanent_uuid: "255d773a238d45cf955ef2805d4d1866" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42971 } } peers { permanent_uuid: "3173464f959549c698515078af36b1e1" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 32961 } }
I20260811 01:44:40.114334 27718 raft_consensus.cc:374] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:40.114452 27718 raft_consensus.cc:729] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 255d773a238d45cf955ef2805d4d1866, State: Initialized, Role: FOLLOWER
I20260811 01:44:40.115072 27718 consensus_queue.cc:260] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcde43dd84354696a1594b38e4dd9bd6" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38577 } } peers { permanent_uuid: "255d773a238d45cf955ef2805d4d1866" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42971 } } peers { permanent_uuid: "3173464f959549c698515078af36b1e1" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 32961 } }
I20260811 01:44:40.116334 27541 heartbeater.cc:503] Master 127.21.214.126:45319 was elected leader, sending a full tablet report...
I20260811 01:44:40.116997 27718 ts_tablet_manager.cc:1428] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866: Time spent starting tablet: real 0.011s user 0.006s sys 0.005s
I20260811 01:44:40.118198 27716 ts_tablet_manager.cc:1428] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6: Time spent starting tablet: real 0.018s user 0.016s sys 0.001s
I20260811 01:44:40.120595 27720 raft_consensus.cc:348] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcde43dd84354696a1594b38e4dd9bd6" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38577 } } peers { permanent_uuid: "255d773a238d45cf955ef2805d4d1866" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42971 } } peers { permanent_uuid: "3173464f959549c698515078af36b1e1" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 32961 } }
I20260811 01:44:40.121224 27720 raft_consensus.cc:374] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:40.121353 27720 raft_consensus.cc:729] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3173464f959549c698515078af36b1e1, State: Initialized, Role: FOLLOWER
I20260811 01:44:40.122125 27720 consensus_queue.cc:260] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcde43dd84354696a1594b38e4dd9bd6" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38577 } } peers { permanent_uuid: "255d773a238d45cf955ef2805d4d1866" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42971 } } peers { permanent_uuid: "3173464f959549c698515078af36b1e1" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 32961 } }
I20260811 01:44:40.123297 27692 heartbeater.cc:503] Master 127.21.214.126:45319 was elected leader, sending a full tablet report...
I20260811 01:44:40.123997 27720 ts_tablet_manager.cc:1428] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
W20260811 01:44:40.180294 27542 tablet.cc:2369] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:44:40.223524 27693 tablet.cc:2369] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:44:40.295612 27727 raft_consensus.cc:482] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:44:40.295967 27727 raft_consensus.cc:504] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcde43dd84354696a1594b38e4dd9bd6" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38577 } } peers { permanent_uuid: "255d773a238d45cf955ef2805d4d1866" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42971 } } peers { permanent_uuid: "3173464f959549c698515078af36b1e1" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 32961 } }
I20260811 01:44:40.297930 27727 leader_election.cc:288] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcde43dd84354696a1594b38e4dd9bd6 (127.21.214.65:38577), 255d773a238d45cf955ef2805d4d1866 (127.21.214.66:42971)
I20260811 01:44:40.304526 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.298522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:57731 (local address 127.21.214.65:38577)
0811 01:44:40.298862 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:40.298872 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:40.298911 (+ 39us) server_negotiation.cc:407] Connection header received
0811 01:44:40.298996 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:40.299000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:40.299064 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:40.299201 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:40.299826 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.300684 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.302605 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.303468 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.303683 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:40.303775 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:40.304085 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 01:44:40.304228 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 01:44:40.304558 27695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.298394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:38577 (local address 127.21.214.67:57731)
0811 01:44:40.298605 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:40.298628 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:40.298771 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:40.299419 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:40.299423 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:40.299456 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:40.299645 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:40.299657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.300864 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.300870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:40.302418 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:40.302431 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.303644 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.303653 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:40.303882 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:40.303912 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 01:44:40.304053 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 01:44:40.304223 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":1}
I20260811 01:44:40.304890 27712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.299050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:38273 (local address 127.21.214.66:42971)
0811 01:44:40.299295 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:40.299302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:40.299783 (+ 481us) server_negotiation.cc:407] Connection header received
0811 01:44:40.299978 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:40.299988 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:40.300045 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:40.300144 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:40.300679 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.301467 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.303159 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.303931 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.304110 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:40.304184 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:40.304490 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 01:44:40.304619 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 01:44:40.304943 27728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.298846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:42971 (local address 127.21.214.67:38273)
0811 01:44:40.299660 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:40.299675 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:40.299826 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:40.300160 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:40.300164 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:40.300186 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:40.300505 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:40.300513 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.301649 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.301653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:40.302991 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:40.303002 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.304152 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.304158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:40.304322 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:40.304346 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 01:44:40.304462 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 01:44:40.304591 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":468,"threads_started":1}
I20260811 01:44:40.306250 27337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb0de64da56b4c719e1e077312f45c08" candidate_uuid: "3173464f959549c698515078af36b1e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcde43dd84354696a1594b38e4dd9bd6" is_pre_election: true
I20260811 01:44:40.306281 27491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb0de64da56b4c719e1e077312f45c08" candidate_uuid: "3173464f959549c698515078af36b1e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255d773a238d45cf955ef2805d4d1866" is_pre_election: true
I20260811 01:44:40.306893 27491 raft_consensus.cc:2454] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3173464f959549c698515078af36b1e1 in term 0.
I20260811 01:44:40.306902 27337 raft_consensus.cc:2454] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3173464f959549c698515078af36b1e1 in term 0.
I20260811 01:44:40.307663 27565 leader_election.cc:302] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3173464f959549c698515078af36b1e1, bcde43dd84354696a1594b38e4dd9bd6; no voters:
I20260811 01:44:40.308058 27727 raft_consensus.cc:2781] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:44:40.308255 27727 raft_consensus.cc:482] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:44:40.308399 27727 raft_consensus.cc:3037] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:40.311064 27727 raft_consensus.cc:504] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcde43dd84354696a1594b38e4dd9bd6" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38577 } } peers { permanent_uuid: "255d773a238d45cf955ef2805d4d1866" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42971 } } peers { permanent_uuid: "3173464f959549c698515078af36b1e1" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 32961 } }
I20260811 01:44:40.311746 27727 leader_election.cc:288] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [CANDIDATE]: Term 1 election: Requested vote from peers bcde43dd84354696a1594b38e4dd9bd6 (127.21.214.65:38577), 255d773a238d45cf955ef2805d4d1866 (127.21.214.66:42971)
I20260811 01:44:40.312309 27337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb0de64da56b4c719e1e077312f45c08" candidate_uuid: "3173464f959549c698515078af36b1e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcde43dd84354696a1594b38e4dd9bd6"
I20260811 01:44:40.312425 27491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb0de64da56b4c719e1e077312f45c08" candidate_uuid: "3173464f959549c698515078af36b1e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "255d773a238d45cf955ef2805d4d1866"
I20260811 01:44:40.312572 27337 raft_consensus.cc:3037] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:40.312655 27491 raft_consensus.cc:3037] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:40.316097 27337 raft_consensus.cc:2454] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3173464f959549c698515078af36b1e1 in term 1.
I20260811 01:44:40.316097 27491 raft_consensus.cc:2454] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3173464f959549c698515078af36b1e1 in term 1.
I20260811 01:44:40.316577 27565 leader_election.cc:302] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3173464f959549c698515078af36b1e1, bcde43dd84354696a1594b38e4dd9bd6; no voters:
I20260811 01:44:40.316943 27727 raft_consensus.cc:2781] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:40.317499 27727 raft_consensus.cc:686] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [term 1 LEADER]: Becoming Leader. State: Replica: 3173464f959549c698515078af36b1e1, State: Running, Role: LEADER
I20260811 01:44:40.318050 27727 consensus_queue.cc:237] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcde43dd84354696a1594b38e4dd9bd6" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38577 } } peers { permanent_uuid: "255d773a238d45cf955ef2805d4d1866" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42971 } } peers { permanent_uuid: "3173464f959549c698515078af36b1e1" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 32961 } }
I20260811 01:44:40.323402 27169 catalog_manager.cc:5495] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3173464f959549c698515078af36b1e1 (127.21.214.67). New cstate: current_term: 1 leader_uuid: "3173464f959549c698515078af36b1e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcde43dd84354696a1594b38e4dd9bd6" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "255d773a238d45cf955ef2805d4d1866" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3173464f959549c698515078af36b1e1" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 32961 } health_report { overall_health: HEALTHY } } }
I20260811 01:44:40.365384 27711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.358707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39926 (local address 127.21.214.67:32961)
0811 01:44:40.358965 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:40.358970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:40.358990 (+ 20us) server_negotiation.cc:407] Connection header received
0811 01:44:40.359206 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:40.359210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:40.359298 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:40.359443 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:44:40.360019 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.361008 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.362729 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.363191 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.363304 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:40.363853 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:44:40.364522 (+ 669us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:44:40.364727 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:40.364884 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 01:44:40.365004 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260811 01:44:40.368141 27388 tablet.cc:2369] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:44:40.374233 27491 raft_consensus.cc:1261] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866 [term 1 FOLLOWER]: Refusing update from remote peer 3173464f959549c698515078af36b1e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:40.374313 27337 raft_consensus.cc:1261] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6 [term 1 FOLLOWER]: Refusing update from remote peer 3173464f959549c698515078af36b1e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:40.375363 27727 consensus_queue.cc:1035] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "255d773a238d45cf955ef2805d4d1866" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:40.375716 27729 consensus_queue.cc:1035] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcde43dd84354696a1594b38e4dd9bd6" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:40.388275 27732 mvcc.cc:205] Tried to move back new op lower bound from 7317146338801176576 to 7317146338586529792. Current Snapshot: MvccSnapshot[applied={T|T < 7317146338801176576}]
I20260811 01:44:40.404907 27169 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:44:40.405282 27169 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:44:40.408874 27169 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1663b7097a04c7babecf998bf825985: Sending DeleteTablet for 3 replicas of tablet eb0de64da56b4c719e1e077312f45c08
I20260811 01:44:40.410133 27311 tablet_service.cc:1514] Processing DeleteTablet for tablet eb0de64da56b4c719e1e077312f45c08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:40 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:42862
I20260811 01:44:40.410234 27467 tablet_service.cc:1514] Processing DeleteTablet for tablet eb0de64da56b4c719e1e077312f45c08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:40 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:53860
I20260811 01:44:40.410243 27618 tablet_service.cc:1514] Processing DeleteTablet for tablet eb0de64da56b4c719e1e077312f45c08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:40 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:39912
I20260811 01:44:40.413192 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 27242
I20260811 01:44:40.441592 27744 tablet_replica.cc:330] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1: stopping tablet replica
I20260811 01:44:40.442160 27744 raft_consensus.cc:2229] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:44:40.442915 27744 raft_consensus.cc:2258] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:40.449366 27742 tablet_replica.cc:330] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866: stopping tablet replica
I20260811 01:44:40.449852 27742 raft_consensus.cc:2229] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:44:40.450424 27742 raft_consensus.cc:2258] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 11 01:44:40 dist-test-slave-9gcw krb5kdc[27121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:44:40.453058 27744 ts_tablet_manager.cc:1905] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:40.461925 27742 ts_tablet_manager.cc:1905] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:40.465721 27743 tablet_replica.cc:330] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6: stopping tablet replica
I20260811 01:44:40.466236 27743 raft_consensus.cc:2229] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:44:40.466840 27743 raft_consensus.cc:2258] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:40.470796 27743 ts_tablet_manager.cc:1905] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:40.472229 27744 ts_tablet_manager.cc:1918] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:44:40.472430 27744 log.cc:1196] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/eb0de64da56b4c719e1e077312f45c08
I20260811 01:44:40.473026 27744 ts_tablet_manager.cc:1939] T eb0de64da56b4c719e1e077312f45c08 P 3173464f959549c698515078af36b1e1: Deleting consensus metadata
I20260811 01:44:40.475167 27154 catalog_manager.cc:4841] TS 3173464f959549c698515078af36b1e1 (127.21.214.67:32961): tablet eb0de64da56b4c719e1e077312f45c08 (table test-table [id=1ab3133f203f4e11b5ada7c4c3288203]) successfully deleted
I20260811 01:44:40.478585 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.414828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42878 (local address 127.21.214.65:38577)
0811 01:44:40.415023 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:40.415031 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:40.438372 (+ 23341us) server_negotiation.cc:407] Connection header received
0811 01:44:40.439004 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:40.439011 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:40.439109 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:40.439192 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:40.443379 (+ 4187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.444634 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.450842 (+ 6208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.451572 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.451687 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:40.468647 (+ 16960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:40.468676 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:40.468695 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:40.468772 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:40.473691 (+ 4919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:40.474826 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:40.474838 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:40.474844 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:40.475030 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:40.475665 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:40.475672 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:40.475674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:40.476177 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:40.476558 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:40.477897 (+ 1339us) server_negotiation.cc:299] Negotiation successful
0811 01:44:40.478169 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 01:44:40.479725 27381 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:40.479851 27742 ts_tablet_manager.cc:1918] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:44:40.480023 27742 log.cc:1196] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/eb0de64da56b4c719e1e077312f45c08
I20260811 01:44:40.480728 27742 ts_tablet_manager.cc:1939] T eb0de64da56b4c719e1e077312f45c08 P 255d773a238d45cf955ef2805d4d1866: Deleting consensus metadata
I20260811 01:44:40.484180 27152 catalog_manager.cc:4841] TS 255d773a238d45cf955ef2805d4d1866 (127.21.214.66:42971): tablet eb0de64da56b4c719e1e077312f45c08 (table test-table [id=1ab3133f203f4e11b5ada7c4c3288203]) successfully deleted
I20260811 01:44:40.588999 27743 ts_tablet_manager.cc:1918] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:44:40.589357 27743 log.cc:1196] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/eb0de64da56b4c719e1e077312f45c08
I20260811 01:44:40.589779 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 27242
I20260811 01:44:40.590034 27743 ts_tablet_manager.cc:1939] T eb0de64da56b4c719e1e077312f45c08 P bcde43dd84354696a1594b38e4dd9bd6: Deleting consensus metadata
I20260811 01:44:40.613548 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 27394
W20260811 01:44:40.613499 27152 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.214.65:38577 (error 108)
I20260811 01:44:40.615731 27748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.614896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:38577 (local address 127.0.0.1:42882)
0811 01:44:40.615357 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:40.615509 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:38577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":230,"threads_started":1}
W20260811 01:44:40.616037 27152 catalog_manager.cc:4584] TS bcde43dd84354696a1594b38e4dd9bd6 (127.21.214.65:38577): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eb0de64da56b4c719e1e077312f45c08: Network error: Client connection negotiation failed: client connection to 127.21.214.65:38577: connect: Connection refused (error 111)
Aug 11 01:44:40 dist-test-slave-9gcw krb5kdc[27121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.66@KRBTEST.COM
I20260811 01:44:40.633690 27712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.614728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53862 (local address 127.21.214.66:42971)
0811 01:44:40.614943 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:40.614951 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:40.615444 (+ 493us) server_negotiation.cc:407] Connection header received
0811 01:44:40.615654 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:40.615660 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:40.615755 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:40.615870 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:40.617519 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.618735 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.620091 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.620728 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.620886 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:40.625986 (+ 5100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:40.626008 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:40.626033 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:40.626084 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:40.630199 (+ 4115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:40.631262 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:40.631273 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:40.631277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:40.631451 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:40.631926 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:40.631930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:40.631933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:40.632423 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:40.632727 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:40.633005 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 01:44:40.633289 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 01:44:40.634531 27535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:40.670434 27748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.669842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:38577 (local address 127.0.0.1:42896)
0811 01:44:40.670113 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:40.670235 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:38577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 01:44:40.671813 27748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.671297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:38577 (local address 127.0.0.1:42902)
0811 01:44:40.671491 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:40.671601 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:38577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 01:44:40.730443 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 27394
I20260811 01:44:40.735792 27748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.735344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:38577 (local address 127.0.0.1:42914)
0811 01:44:40.735548 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:40.735644 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:38577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 01:44:40.736861 27748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.736484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:38577 (local address 127.0.0.1:42928)
0811 01:44:40.736635 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:40.736718 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:38577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 01:44:40.758091 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 27547
Aug 11 01:44:40 dist-test-slave-9gcw krb5kdc[27121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.67@KRBTEST.COM
I20260811 01:44:40.777848 27711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.759089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39930 (local address 127.21.214.67:32961)
0811 01:44:40.759310 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:40.759317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:40.759343 (+ 26us) server_negotiation.cc:407] Connection header received
0811 01:44:40.759428 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:40.759433 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:40.759530 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:40.759653 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:40.761087 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.762483 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.763816 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:40.764594 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:40.764768 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:40.770011 (+ 5243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:40.770028 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:40.770050 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:40.770100 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:40.774466 (+ 4366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:40.775367 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:40.775377 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:40.775380 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:40.775598 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:40.776039 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:40.776044 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:40.776048 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:40.776594 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:40.776938 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:40.777105 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 01:44:40.777405 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 01:44:40.778759 27684 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:40.818360 27748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.817532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:38577 (local address 127.0.0.1:42944)
0811 01:44:40.818072 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:40.818210 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:38577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 01:44:40.819839 27748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:40.819375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:38577 (local address 127.0.0.1:42950)
0811 01:44:40.819603 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:40.819687 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:38577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260811 01:44:40.909792 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 27547
I20260811 01:44:40.941452 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 27135
I20260811 01:44:40.942888 27204 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:41.054562 27748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:41.053921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:38577 (local address 127.0.0.1:42954)
0811 01:44:41.054201 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:41.054356 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:38577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 01:44:41.054863 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 27135
2026-08-11T01:44:41Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4809 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T01:44:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:44:41Z Disabled control of system clock
I20260811 01:44:41.138337 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:42823
--webserver_interface=127.21.214.126
--webserver_port=0
--builtin_ntp_servers=127.21.214.84:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:42823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 01:44:41.512176 27756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:41.512516 27756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:41.512714 27756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:41.520330 27756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:44:41.520455 27756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:41.520531 27756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:41.520627 27756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:44:41.520738 27756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:44:41.530462 27756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:42823
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:42823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:41.531956 27756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:41.534029 27756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:41.546906 27761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:41.546975 27764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:41.547026 27762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:41.548396 27756 server_base.cc:1034] running on GCE node
I20260811 01:44:41.549428 27756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:41.551036 27756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:41.552273 27756 hybrid_clock.cc:648] HybridClock initialized: now 1786412681552229 us; error 40 us; skew 500 ppm
I20260811 01:44:41.552702 27756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:44:41.555394 27756 webserver.cc:460] Webserver started at http://127.21.214.126:43617/ using document root <none> and password file <none>
I20260811 01:44:41.556180 27756 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:41.556327 27756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:41.556701 27756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:41.560045 27756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "e1759b3d12e945b1bb6276a9f33eb0e3"
format_stamp: "Formatted at 2026-08-11 01:44:41 on dist-test-slave-9gcw"
server_key: "842be7ee996a1cef85ca092d22173309"
server_key_iv: "6d0db45a6a7e6fa96f21335b080bbe1e"
server_key_version: "encryptionkey@0"
I20260811 01:44:41.560935 27756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "e1759b3d12e945b1bb6276a9f33eb0e3"
format_stamp: "Formatted at 2026-08-11 01:44:41 on dist-test-slave-9gcw"
server_key: "842be7ee996a1cef85ca092d22173309"
server_key_iv: "6d0db45a6a7e6fa96f21335b080bbe1e"
server_key_version: "encryptionkey@0"
I20260811 01:44:41.566740 27756 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 01:44:41.570484 27770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:41.572058 27756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:44:41.572269 27756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "e1759b3d12e945b1bb6276a9f33eb0e3"
format_stamp: "Formatted at 2026-08-11 01:44:41 on dist-test-slave-9gcw"
server_key: "842be7ee996a1cef85ca092d22173309"
server_key_iv: "6d0db45a6a7e6fa96f21335b080bbe1e"
server_key_version: "encryptionkey@0"
I20260811 01:44:41.572503 27756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:41.578679 27756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:41.579485 27756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:41.579849 27756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:41.599355 27756 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:42823
I20260811 01:44:41.599406 27827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:42823 every 8 connection(s)
I20260811 01:44:41.600801 27756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:44:41.604429 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 27756
I20260811 01:44:41.604640 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:44:41.605556 22361 external_mini_cluster.cc:1428] Setting key ae01cdc4b34036c5afe02307083d1923
I20260811 01:44:41.605573 27828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:41.614521 27832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:41.607863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41252 (local address 127.21.214.126:42823)
0811 01:44:41.608922 (+ 1059us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:41.608934 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:41.608979 (+ 45us) server_negotiation.cc:407] Connection header received
0811 01:44:41.610707 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:41.610732 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:41.611166 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:41.611585 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:41.612734 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:41.612743 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:41.612757 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:41.612771 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:41.612839 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:41.612858 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:41.613124 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:41.613317 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 01:44:41.613584 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":392,"threads_started":1}
I20260811 01:44:41.620499 27828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3: Bootstrap starting.
I20260811 01:44:41.623994 27828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:41.625222 27828 log.cc:824] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:41.628686 27828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3: No bootstrap required, opened a new log
I20260811 01:44:41.636253 27828 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1759b3d12e945b1bb6276a9f33eb0e3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42823 } }
I20260811 01:44:41.636754 27828 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:41.636889 27828 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1759b3d12e945b1bb6276a9f33eb0e3, State: Initialized, Role: FOLLOWER
I20260811 01:44:41.637620 27828 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1759b3d12e945b1bb6276a9f33eb0e3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42823 } }
I20260811 01:44:41.637846 27828 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:44:41.637950 27828 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:44:41.638087 27828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:41.640301 27828 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1759b3d12e945b1bb6276a9f33eb0e3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42823 } }
I20260811 01:44:41.640831 27828 leader_election.cc:302] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1759b3d12e945b1bb6276a9f33eb0e3; no voters:
I20260811 01:44:41.641485 27828 leader_election.cc:288] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:44:41.641721 27834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:41.642454 27834 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3 [term 1 LEADER]: Becoming Leader. State: Replica: e1759b3d12e945b1bb6276a9f33eb0e3, State: Running, Role: LEADER
I20260811 01:44:41.643146 27834 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1759b3d12e945b1bb6276a9f33eb0e3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42823 } }
I20260811 01:44:41.643898 27828 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:44:41.646967 27836 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1759b3d12e945b1bb6276a9f33eb0e3. Latest consensus state: current_term: 1 leader_uuid: "e1759b3d12e945b1bb6276a9f33eb0e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1759b3d12e945b1bb6276a9f33eb0e3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42823 } } }
I20260811 01:44:41.647445 27836 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3 [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:41.646993 27835 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1759b3d12e945b1bb6276a9f33eb0e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1759b3d12e945b1bb6276a9f33eb0e3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42823 } } }
I20260811 01:44:41.647608 27835 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3 [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:41.649525 27843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:44:41.655284 27843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:44:41.665581 27843 catalog_manager.cc:1347] Generated new cluster ID: 5c8d1862ad82444186b1fd00702a8c5a
I20260811 01:44:41.665731 27843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:44:41.687314 27843 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:44:41.687824 27843 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:44:41.696302 27843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3: Generated new TSK 0
I20260811 01:44:41.697345 27843 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:44:41.702831 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42823
--builtin_ntp_servers=127.21.214.84:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 01:44:42.079452 27856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:42.079769 27856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:42.079938 27856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:42.087375 27856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:42.087495 27856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:42.087679 27856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:44:42.097357 27856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:42.098970 27856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:42.101055 27856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:42.115620 27866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:42.115574 27865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:42.115655 27868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:42.117208 27856 server_base.cc:1034] running on GCE node
I20260811 01:44:42.117899 27856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:42.119059 27856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:42.120325 27856 hybrid_clock.cc:648] HybridClock initialized: now 1786412682120249 us; error 64 us; skew 500 ppm
I20260811 01:44:42.120749 27856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:44:42.123694 27856 webserver.cc:460] Webserver started at http://127.21.214.65:33725/ using document root <none> and password file <none>
I20260811 01:44:42.124545 27856 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:42.124711 27856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:42.125135 27856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:42.128721 27856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "f601565fb6874a218f02dd95489e8a77"
format_stamp: "Formatted at 2026-08-11 01:44:42 on dist-test-slave-9gcw"
server_key: "009608d55e5a74463a1960b48bc73dd4"
server_key_iv: "515724dcbbc44e08f0724f59df05870d"
server_key_version: "encryptionkey@0"
I20260811 01:44:42.129805 27856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "f601565fb6874a218f02dd95489e8a77"
format_stamp: "Formatted at 2026-08-11 01:44:42 on dist-test-slave-9gcw"
server_key: "009608d55e5a74463a1960b48bc73dd4"
server_key_iv: "515724dcbbc44e08f0724f59df05870d"
server_key_version: "encryptionkey@0"
I20260811 01:44:42.136349 27856 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 01:44:42.140564 27874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:42.142416 27856 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 01:44:42.142647 27856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "f601565fb6874a218f02dd95489e8a77"
format_stamp: "Formatted at 2026-08-11 01:44:42 on dist-test-slave-9gcw"
server_key: "009608d55e5a74463a1960b48bc73dd4"
server_key_iv: "515724dcbbc44e08f0724f59df05870d"
server_key_version: "encryptionkey@0"
I20260811 01:44:42.142925 27856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:42.149883 27856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:42.150779 27856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:42.151099 27856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:42.152100 27856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:42.153693 27856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:42.153813 27856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:42.153962 27856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:42.154067 27856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:42.188807 27856 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:45061
I20260811 01:44:42.188854 27999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:45061 every 8 connection(s)
I20260811 01:44:42.190260 27856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:44:42.192260 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 27856
I20260811 01:44:42.192487 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:44:42.193224 22361 external_mini_cluster.cc:1428] Setting key 2abc22ff74705e6c10334a9ea1ed17fe
I20260811 01:44:42.196798 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42823
--builtin_ntp_servers=127.21.214.84:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 01:44:42.200133 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:42.193384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:58891 (local address 127.21.214.126:42823)
0811 01:44:42.193905 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:42.193912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:42.195570 (+ 1658us) server_negotiation.cc:407] Connection header received
0811 01:44:42.197654 (+ 2084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:42.197663 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:42.197736 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:42.197867 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:42.199373 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:42.199377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:42.199381 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:42.199397 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:42.199442 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:42.199454 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:42.199558 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:42.199845 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 01:44:42.199906 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":256,"threads_started":1}
I20260811 01:44:42.201586 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:42.194077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:42823 (local address 127.21.214.65:58891)
0811 01:44:42.195388 (+ 1311us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:42.195443 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:42.197368 (+ 1925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:42.198077 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:42.198085 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:42.198183 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:42.198948 (+ 765us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:44:42.198961 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:42.199028 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:44:42.199155 (+ 127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:42.199557 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:42.199570 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:42.199581 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 01:44:42.199889 (+ 308us) client_negotiation.cc:240] Negotiation successful
0811 01:44:42.200184 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":365,"threads_started":1}
I20260811 01:44:42.203279 28000 heartbeater.cc:347] Connected to a master server at 127.21.214.126:42823
I20260811 01:44:42.203677 28000 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:42.204468 28000 heartbeater.cc:511] Master 127.21.214.126:42823 requested a full tablet report, sending...
I20260811 01:44:42.207166 27789 ts_manager.cc:194] Registered new tserver with Master: f601565fb6874a218f02dd95489e8a77 (127.21.214.65:45061)
W20260811 01:44:42.545289 28004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:42.545641 28004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:42.545814 28004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:42.553618 28004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:42.553741 28004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:42.553913 28004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:44:42.563454 28004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:42.564950 28004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:42.566838 28004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:42.581003 28009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:42.581256 28012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:42.581578 28014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:42.582392 28004 server_base.cc:1034] running on GCE node
I20260811 01:44:42.583192 28004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:42.584406 28004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:42.585798 28004 hybrid_clock.cc:648] HybridClock initialized: now 1786412682585689 us; error 46 us; skew 500 ppm
I20260811 01:44:42.586198 28004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:44:42.588873 28004 webserver.cc:460] Webserver started at http://127.21.214.66:45861/ using document root <none> and password file <none>
I20260811 01:44:42.589743 28004 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:42.589897 28004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:42.590278 28004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:42.593757 28004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "4008534ac65f4eee828e41b437e45c2b"
format_stamp: "Formatted at 2026-08-11 01:44:42 on dist-test-slave-9gcw"
server_key: "feea9fd13a44fe39d552893208489bab"
server_key_iv: "4c4c27eefb2d82e13f4a02b014f2146a"
server_key_version: "encryptionkey@0"
I20260811 01:44:42.594682 28004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "4008534ac65f4eee828e41b437e45c2b"
format_stamp: "Formatted at 2026-08-11 01:44:42 on dist-test-slave-9gcw"
server_key: "feea9fd13a44fe39d552893208489bab"
server_key_iv: "4c4c27eefb2d82e13f4a02b014f2146a"
server_key_version: "encryptionkey@0"
I20260811 01:44:42.600942 28004 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260811 01:44:42.605044 28022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:42.606542 28004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:44:42.606789 28004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "4008534ac65f4eee828e41b437e45c2b"
format_stamp: "Formatted at 2026-08-11 01:44:42 on dist-test-slave-9gcw"
server_key: "feea9fd13a44fe39d552893208489bab"
server_key_iv: "4c4c27eefb2d82e13f4a02b014f2146a"
server_key_version: "encryptionkey@0"
I20260811 01:44:42.607039 28004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:42.613662 28004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:42.614756 28004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:42.615069 28004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:42.616122 28004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:42.617764 28004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:42.617905 28004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:42.618069 28004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:42.618173 28004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:42.656041 28004 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:46729
I20260811 01:44:42.656095 28154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:46729 every 8 connection(s)
I20260811 01:44:42.657734 28004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:44:42.665303 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 28004
I20260811 01:44:42.665488 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:44:42.666102 22361 external_mini_cluster.cc:1428] Setting key d4c0b5fb106ed413ff78a3182262b181
I20260811 01:44:42.666580 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:42.660512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:43057 (local address 127.21.214.126:42823)
0811 01:44:42.660687 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:42.660691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:42.662227 (+ 1536us) server_negotiation.cc:407] Connection header received
0811 01:44:42.663924 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:42.663928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:42.664004 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:42.664107 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:42.665819 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:42.665824 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:42.665828 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:42.665845 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:42.665881 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:42.665894 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:42.666016 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:42.666272 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 01:44:42.666346 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 01:44:42.667842 28157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:42.661027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:42823 (local address 127.21.214.66:43057)
0811 01:44:42.662046 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:42.662083 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:42.663638 (+ 1555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:42.664316 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:42.664324 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:42.664417 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:42.665412 (+ 995us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:44:42.665417 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:42.665474 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:44:42.665602 (+ 128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:42.666007 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:42.666018 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:42.666026 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 01:44:42.666350 (+ 324us) client_negotiation.cc:240] Negotiation successful
0811 01:44:42.666628 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":301,"threads_started":1}
I20260811 01:44:42.669394 28155 heartbeater.cc:347] Connected to a master server at 127.21.214.126:42823
I20260811 01:44:42.669611 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42823
--builtin_ntp_servers=127.21.214.84:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 01:44:42.669776 28155 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:42.670429 28155 heartbeater.cc:511] Master 127.21.214.126:42823 requested a full tablet report, sending...
I20260811 01:44:42.672053 27789 ts_manager.cc:194] Registered new tserver with Master: 4008534ac65f4eee828e41b437e45c2b (127.21.214.66:46729)
W20260811 01:44:43.024997 28158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:43.025374 28158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:43.025549 28158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:43.033025 28158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:43.033214 28158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:43.033407 28158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:44:43.042900 28158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:43.044435 28158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:43.046571 28158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:43.065428 28163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:43.065492 28164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:43.065467 28167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:43.066260 28158 server_base.cc:1034] running on GCE node
I20260811 01:44:43.066983 28158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:43.068053 28158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:43.069272 28158 hybrid_clock.cc:648] HybridClock initialized: now 1786412683069203 us; error 55 us; skew 500 ppm
I20260811 01:44:43.069640 28158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:44:43.072309 28158 webserver.cc:460] Webserver started at http://127.21.214.67:38929/ using document root <none> and password file <none>
I20260811 01:44:43.073115 28158 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:43.073325 28158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:43.073704 28158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:43.076987 28158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "64791d742cd2420b929e23514ca8785b"
format_stamp: "Formatted at 2026-08-11 01:44:43 on dist-test-slave-9gcw"
server_key: "8f0bcf2bc9a33f21b42e4e19d2d90980"
server_key_iv: "407f6c214776ff16d315d41a2af35870"
server_key_version: "encryptionkey@0"
I20260811 01:44:43.077915 28158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "64791d742cd2420b929e23514ca8785b"
format_stamp: "Formatted at 2026-08-11 01:44:43 on dist-test-slave-9gcw"
server_key: "8f0bcf2bc9a33f21b42e4e19d2d90980"
server_key_iv: "407f6c214776ff16d315d41a2af35870"
server_key_version: "encryptionkey@0"
I20260811 01:44:43.083607 28158 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 01:44:43.087347 28175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:43.088816 28158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 01:44:43.089017 28158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "64791d742cd2420b929e23514ca8785b"
format_stamp: "Formatted at 2026-08-11 01:44:43 on dist-test-slave-9gcw"
server_key: "8f0bcf2bc9a33f21b42e4e19d2d90980"
server_key_iv: "407f6c214776ff16d315d41a2af35870"
server_key_version: "encryptionkey@0"
I20260811 01:44:43.089313 28158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:43.095500 28158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:43.096380 28158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:43.096679 28158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:43.097716 28158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:43.099284 28158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:43.099411 28158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:43.099555 28158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:43.099645 28158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:43.134831 28158 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:45347
I20260811 01:44:43.134930 28301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:45347 every 8 connection(s)
I20260811 01:44:43.136288 28158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:44:43.145087 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.139206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:39051 (local address 127.21.214.126:42823)
0811 01:44:43.139405 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:43.139411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:43.140834 (+ 1423us) server_negotiation.cc:407] Connection header received
0811 01:44:43.142631 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:43.142636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:43.142699 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:43.142797 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:43.144285 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:43.144290 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:43.144293 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:43.144310 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:43.144342 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:43.144354 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:43.144452 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:43.144627 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 01:44:43.144743 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 01:44:43.145459 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 28158
I20260811 01:44:43.145633 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:44:43.146070 28305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.139762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:42823 (local address 127.21.214.67:39051)
0811 01:44:43.140683 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:43.140720 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:43.142336 (+ 1616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:43.143003 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:43.143011 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:43.143101 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:43.143906 (+ 805us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:44:43.143911 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:43.143966 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:44:43.144102 (+ 136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:43.144446 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:43.144454 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:43.144460 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 01:44:43.144675 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 01:44:43.144894 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":273,"threads_started":1}
I20260811 01:44:43.146101 22361 external_mini_cluster.cc:1428] Setting key a521e501e389150b9e046433f8f323aa
I20260811 01:44:43.147670 28303 heartbeater.cc:347] Connected to a master server at 127.21.214.126:42823
I20260811 01:44:43.148054 28303 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:43.148703 28303 heartbeater.cc:511] Master 127.21.214.126:42823 requested a full tablet report, sending...
I20260811 01:44:43.150292 27789 ts_manager.cc:194] Registered new tserver with Master: 64791d742cd2420b929e23514ca8785b (127.21.214.67:45347)
I20260811 01:44:43.162129 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:44:43.170149 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.167142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58490 (local address 127.21.214.126:42823)
0811 01:44:43.167341 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:43.167346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:43.167838 (+ 492us) server_negotiation.cc:407] Connection header received
0811 01:44:43.167979 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:43.167983 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:43.168038 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:43.168133 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:43.169488 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:43.169493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:43.169496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:43.169512 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:43.169556 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:43.169569 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:43.169696 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:43.169857 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 01:44:43.169916 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 01:44:43.174489 27789 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:44:43.181932 27789 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:44:43.201831 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.195615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:46729 (local address 127.0.0.1:34322)
0811 01:44:43.196809 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:43.196875 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:43.197045 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:43.198591 (+ 1546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:43.198603 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:43.198726 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:43.199697 (+ 971us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:44:43.199701 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:43.199732 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:44:43.199829 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:43.200744 (+ 915us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:43.200761 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:43.200789 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 01:44:43.201005 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 01:44:43.201138 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"thread_start_us":644,"threads_started":1}
I20260811 01:44:43.203262 28319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.196070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34322 (local address 127.21.214.66:46729)
0811 01:44:43.198041 (+ 1971us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:43.198052 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:43.198081 (+ 29us) server_negotiation.cc:407] Connection header received
0811 01:44:43.198172 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:43.198178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:43.198355 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:43.198475 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:43.200084 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:43.200099 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:43.200121 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:43.200158 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:43.200536 (+ 378us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:43.200573 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:43.202724 (+ 2151us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:43.202800 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 01:44:43.202924 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1817,"thread_start_us":1707,"threads_started":1}
I20260811 01:44:43.204042 28318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.195994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:45061 (local address 127.0.0.1:48664)
0811 01:44:43.199623 (+ 3629us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:43.199638 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:43.199861 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:43.201777 (+ 1916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:43.201785 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:43.201825 (+ 40us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:43.202704 (+ 879us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:44:43.202710 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:43.202735 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:44:43.202803 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:43.203408 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:43.203416 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:43.203424 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 01:44:43.203555 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 01:44:43.203624 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3463,"thread_start_us":3309,"threads_started":1}
I20260811 01:44:43.204927 28320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.196148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48664 (local address 127.21.214.65:45061)
0811 01:44:43.197648 (+ 1500us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:43.197655 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:43.199762 (+ 2107us) server_negotiation.cc:407] Connection header received
0811 01:44:43.200039 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:43.200048 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:43.200367 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:43.200506 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:43.203010 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:43.203020 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:43.203036 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:43.203057 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:43.203131 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:43.203153 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:43.203335 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:43.204425 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0811 01:44:43.204540 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1349,"thread_start_us":1206,"threads_started":1}
I20260811 01:44:43.206363 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.199752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:45347 (local address 127.0.0.1:37354)
0811 01:44:43.202120 (+ 2368us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:43.202137 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:43.202235 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:43.204528 (+ 2293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:43.204533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:43.204573 (+ 40us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:43.205256 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:44:43.205260 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:43.205277 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:44:43.205323 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:43.205952 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:43.205960 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:43.205966 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 01:44:43.206079 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 01:44:43.206131 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2269,"thread_start_us":2690,"threads_started":1}
I20260811 01:44:43.206638 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.200115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37354 (local address 127.21.214.67:45347)
0811 01:44:43.203735 (+ 3620us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:43.203743 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:43.203785 (+ 42us) server_negotiation.cc:407] Connection header received
0811 01:44:43.203848 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:43.203854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:43.204013 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:43.204206 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:43.205546 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:43.205555 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:43.205569 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:43.205588 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:43.205648 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:43.205671 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:43.205876 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:43.206131 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 01:44:43.206235 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3429,"thread_start_us":3257,"threads_started":1}
I20260811 01:44:43.207818 28080 tablet_service.cc:1467] Processing CreateTablet for tablet 528c0ed8e63642c9b071d1de01ab5fb2 (DEFAULT_TABLE table=test-table [id=7ab458624f2041e6a2c3b744874b4d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:43.209113 28080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 528c0ed8e63642c9b071d1de01ab5fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:43.208775 27929 tablet_service.cc:1467] Processing CreateTablet for tablet 528c0ed8e63642c9b071d1de01ab5fb2 (DEFAULT_TABLE table=test-table [id=7ab458624f2041e6a2c3b744874b4d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:43.210093 27929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 528c0ed8e63642c9b071d1de01ab5fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:43.210669 28000 heartbeater.cc:503] Master 127.21.214.126:42823 was elected leader, sending a full tablet report...
I20260811 01:44:43.213372 28231 tablet_service.cc:1467] Processing CreateTablet for tablet 528c0ed8e63642c9b071d1de01ab5fb2 (DEFAULT_TABLE table=test-table [id=7ab458624f2041e6a2c3b744874b4d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:44:43.214937 28231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 528c0ed8e63642c9b071d1de01ab5fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:43.222550 28323 tablet_bootstrap.cc:492] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77: Bootstrap starting.
I20260811 01:44:43.224730 28324 tablet_bootstrap.cc:492] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b: Bootstrap starting.
I20260811 01:44:43.227694 28323 tablet_bootstrap.cc:654] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:43.228582 28324 tablet_bootstrap.cc:654] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:43.228888 28325 tablet_bootstrap.cc:492] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b: Bootstrap starting.
I20260811 01:44:43.229077 28323 log.cc:824] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:43.229946 28324 log.cc:824] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:43.232504 28325 tablet_bootstrap.cc:654] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:43.233309 28324 tablet_bootstrap.cc:492] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b: No bootstrap required, opened a new log
I20260811 01:44:43.233484 28323 tablet_bootstrap.cc:492] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77: No bootstrap required, opened a new log
I20260811 01:44:43.233616 28324 ts_tablet_manager.cc:1397] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 01:44:43.233788 28323 ts_tablet_manager.cc:1397] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 01:44:43.233867 28325 log.cc:824] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:43.237012 28325 tablet_bootstrap.cc:492] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b: No bootstrap required, opened a new log
I20260811 01:44:43.237314 28325 ts_tablet_manager.cc:1397] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 01:44:43.241226 28324 raft_consensus.cc:348] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4008534ac65f4eee828e41b437e45c2b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46729 } } peers { permanent_uuid: "f601565fb6874a218f02dd95489e8a77" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 45061 } } peers { permanent_uuid: "64791d742cd2420b929e23514ca8785b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45347 } }
I20260811 01:44:43.241809 28324 raft_consensus.cc:374] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:43.241971 28324 raft_consensus.cc:729] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4008534ac65f4eee828e41b437e45c2b, State: Initialized, Role: FOLLOWER
I20260811 01:44:43.242571 28324 consensus_queue.cc:260] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4008534ac65f4eee828e41b437e45c2b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46729 } } peers { permanent_uuid: "f601565fb6874a218f02dd95489e8a77" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 45061 } } peers { permanent_uuid: "64791d742cd2420b929e23514ca8785b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45347 } }
I20260811 01:44:43.243664 28155 heartbeater.cc:503] Master 127.21.214.126:42823 was elected leader, sending a full tablet report...
I20260811 01:44:43.244154 28323 raft_consensus.cc:348] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4008534ac65f4eee828e41b437e45c2b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46729 } } peers { permanent_uuid: "f601565fb6874a218f02dd95489e8a77" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 45061 } } peers { permanent_uuid: "64791d742cd2420b929e23514ca8785b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45347 } }
I20260811 01:44:43.244691 28323 raft_consensus.cc:374] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:43.244848 28323 raft_consensus.cc:729] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f601565fb6874a218f02dd95489e8a77, State: Initialized, Role: FOLLOWER
I20260811 01:44:43.245358 28324 ts_tablet_manager.cc:1428] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260811 01:44:43.245635 28323 consensus_queue.cc:260] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4008534ac65f4eee828e41b437e45c2b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46729 } } peers { permanent_uuid: "f601565fb6874a218f02dd95489e8a77" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 45061 } } peers { permanent_uuid: "64791d742cd2420b929e23514ca8785b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45347 } }
I20260811 01:44:43.247468 28323 ts_tablet_manager.cc:1428] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260811 01:44:43.247929 28325 raft_consensus.cc:348] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4008534ac65f4eee828e41b437e45c2b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46729 } } peers { permanent_uuid: "f601565fb6874a218f02dd95489e8a77" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 45061 } } peers { permanent_uuid: "64791d742cd2420b929e23514ca8785b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45347 } }
I20260811 01:44:43.248368 28325 raft_consensus.cc:374] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:43.248481 28325 raft_consensus.cc:729] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64791d742cd2420b929e23514ca8785b, State: Initialized, Role: FOLLOWER
I20260811 01:44:43.249081 28325 consensus_queue.cc:260] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4008534ac65f4eee828e41b437e45c2b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46729 } } peers { permanent_uuid: "f601565fb6874a218f02dd95489e8a77" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 45061 } } peers { permanent_uuid: "64791d742cd2420b929e23514ca8785b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45347 } }
I20260811 01:44:43.250638 28303 heartbeater.cc:503] Master 127.21.214.126:42823 was elected leader, sending a full tablet report...
I20260811 01:44:43.250989 28325 ts_tablet_manager.cc:1428] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260811 01:44:43.388188 28304 tablet.cc:2369] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:44:43.410394 28156 tablet.cc:2369] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:44:43.424326 28331 raft_consensus.cc:482] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:44:43.424641 28331 raft_consensus.cc:504] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4008534ac65f4eee828e41b437e45c2b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46729 } } peers { permanent_uuid: "f601565fb6874a218f02dd95489e8a77" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 45061 } } peers { permanent_uuid: "64791d742cd2420b929e23514ca8785b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45347 } }
I20260811 01:44:43.426514 28331 leader_election.cc:288] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4008534ac65f4eee828e41b437e45c2b (127.21.214.66:46729), 64791d742cd2420b929e23514ca8785b (127.21.214.67:45347)
I20260811 01:44:43.430336 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.427126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:60869 (local address 127.21.214.67:45347)
0811 01:44:43.427481 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:43.427488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:43.428051 (+ 563us) server_negotiation.cc:407] Connection header received
0811 01:44:43.428284 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:43.428290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:43.428359 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:43.428487 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:43.429656 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:43.429662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:43.429666 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:43.429697 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:43.429745 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:43.429757 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:43.429896 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:43.429974 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 01:44:43.430046 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227}
I20260811 01:44:43.430465 28319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.427267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:47663 (local address 127.21.214.66:46729)
0811 01:44:43.427454 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:43.427460 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:43.428507 (+ 1047us) server_negotiation.cc:407] Connection header received
0811 01:44:43.428656 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:43.428665 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:43.428714 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:43.428828 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:43.429768 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:43.429774 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:43.429778 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:43.429804 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:43.429836 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:43.429849 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:43.429973 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:43.430074 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 01:44:43.430143 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 01:44:43.430611 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.426902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:45347 (local address 127.21.214.65:60869)
0811 01:44:43.427922 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:43.427940 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:43.428117 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:43.428512 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:43.428517 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:43.428546 (+ 29us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:43.429388 (+ 842us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:44:43.429393 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:43.429413 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:44:43.429478 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:43.429859 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:43.429864 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:43.429868 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 01:44:43.430039 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 01:44:43.430096 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"thread_start_us":1194,"threads_started":1}
I20260811 01:44:43.430672 28333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.426968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:46729 (local address 127.21.214.65:47663)
0811 01:44:43.428390 (+ 1422us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:43.428405 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:43.428544 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:43.428855 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:43.428859 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:43.428881 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:43.429533 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:44:43.429536 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:43.429550 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:44:43.429587 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:43.429947 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:43.429951 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:43.429957 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 01:44:43.430119 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 01:44:43.430173 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1311,"thread_start_us":662,"threads_started":1}
I20260811 01:44:43.431802 28253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "528c0ed8e63642c9b071d1de01ab5fb2" candidate_uuid: "f601565fb6874a218f02dd95489e8a77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64791d742cd2420b929e23514ca8785b" is_pre_election: true
I20260811 01:44:43.431803 28102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "528c0ed8e63642c9b071d1de01ab5fb2" candidate_uuid: "f601565fb6874a218f02dd95489e8a77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4008534ac65f4eee828e41b437e45c2b" is_pre_election: true
I20260811 01:44:43.432322 28253 raft_consensus.cc:2454] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f601565fb6874a218f02dd95489e8a77 in term 0.
I20260811 01:44:43.432331 28102 raft_consensus.cc:2454] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f601565fb6874a218f02dd95489e8a77 in term 0.
I20260811 01:44:43.433007 27875 leader_election.cc:302] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64791d742cd2420b929e23514ca8785b, f601565fb6874a218f02dd95489e8a77; no voters:
I20260811 01:44:43.433418 28331 raft_consensus.cc:2781] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:44:43.433580 28331 raft_consensus.cc:482] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:44:43.433702 28331 raft_consensus.cc:3037] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:43.436028 28331 raft_consensus.cc:504] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4008534ac65f4eee828e41b437e45c2b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46729 } } peers { permanent_uuid: "f601565fb6874a218f02dd95489e8a77" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 45061 } } peers { permanent_uuid: "64791d742cd2420b929e23514ca8785b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45347 } }
I20260811 01:44:43.436700 28331 leader_election.cc:288] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [CANDIDATE]: Term 1 election: Requested vote from peers 4008534ac65f4eee828e41b437e45c2b (127.21.214.66:46729), 64791d742cd2420b929e23514ca8785b (127.21.214.67:45347)
I20260811 01:44:43.437289 28253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "528c0ed8e63642c9b071d1de01ab5fb2" candidate_uuid: "f601565fb6874a218f02dd95489e8a77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64791d742cd2420b929e23514ca8785b"
I20260811 01:44:43.437358 28102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "528c0ed8e63642c9b071d1de01ab5fb2" candidate_uuid: "f601565fb6874a218f02dd95489e8a77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4008534ac65f4eee828e41b437e45c2b"
I20260811 01:44:43.437517 28253 raft_consensus.cc:3037] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:43.437608 28102 raft_consensus.cc:3037] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:43.440047 28253 raft_consensus.cc:2454] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f601565fb6874a218f02dd95489e8a77 in term 1.
I20260811 01:44:43.440173 28102 raft_consensus.cc:2454] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f601565fb6874a218f02dd95489e8a77 in term 1.
I20260811 01:44:43.440601 27875 leader_election.cc:302] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64791d742cd2420b929e23514ca8785b, f601565fb6874a218f02dd95489e8a77; no voters:
I20260811 01:44:43.440945 28331 raft_consensus.cc:2781] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:43.441498 28331 raft_consensus.cc:686] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [term 1 LEADER]: Becoming Leader. State: Replica: f601565fb6874a218f02dd95489e8a77, State: Running, Role: LEADER
I20260811 01:44:43.442014 28331 consensus_queue.cc:237] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4008534ac65f4eee828e41b437e45c2b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46729 } } peers { permanent_uuid: "f601565fb6874a218f02dd95489e8a77" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 45061 } } peers { permanent_uuid: "64791d742cd2420b929e23514ca8785b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45347 } }
W20260811 01:44:43.442987 28001 tablet.cc:2369] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:44:43.447088 27789 catalog_manager.cc:5495] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 reported cstate change: term changed from 0 to 1, leader changed from <none> to f601565fb6874a218f02dd95489e8a77 (127.21.214.65). New cstate: current_term: 1 leader_uuid: "f601565fb6874a218f02dd95489e8a77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4008534ac65f4eee828e41b437e45c2b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f601565fb6874a218f02dd95489e8a77" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 45061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64791d742cd2420b929e23514ca8785b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260811 01:44:43.508343 28320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.505869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48678 (local address 127.21.214.65:45061)
0811 01:44:43.506086 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:43.506093 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:43.506276 (+ 183us) server_negotiation.cc:407] Connection header received
0811 01:44:43.506574 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:43.506579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:43.506667 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:43.506793 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:43.507698 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:43.507702 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:43.507705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:43.507721 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:43.507753 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:43.507765 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:43.507921 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:43.508014 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 01:44:43.508086 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 01:44:43.516983 28102 raft_consensus.cc:1261] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b [term 1 FOLLOWER]: Refusing update from remote peer f601565fb6874a218f02dd95489e8a77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:43.516988 28253 raft_consensus.cc:1261] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b [term 1 FOLLOWER]: Refusing update from remote peer f601565fb6874a218f02dd95489e8a77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:44:43.518018 28335 consensus_queue.cc:1035] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64791d742cd2420b929e23514ca8785b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:43.518376 28331 consensus_queue.cc:1035] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4008534ac65f4eee828e41b437e45c2b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:43.545876 27788 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:44:43.546191 27788 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:44:43.550416 27788 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e1759b3d12e945b1bb6276a9f33eb0e3: Sending DeleteTablet for 3 replicas of tablet 528c0ed8e63642c9b071d1de01ab5fb2
I20260811 01:44:43.551800 28080 tablet_service.cc:1514] Processing DeleteTablet for tablet 528c0ed8e63642c9b071d1de01ab5fb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:43 UTC) from {username='slave'} at 127.0.0.1:34322
I20260811 01:44:43.551883 28231 tablet_service.cc:1514] Processing DeleteTablet for tablet 528c0ed8e63642c9b071d1de01ab5fb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:43 UTC) from {username='slave'} at 127.0.0.1:37354
I20260811 01:44:43.552839 28350 tablet_replica.cc:330] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b: stopping tablet replica
I20260811 01:44:43.553035 28351 tablet_replica.cc:330] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b: stopping tablet replica
I20260811 01:44:43.553444 28350 raft_consensus.cc:2229] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:44:43.553457 28351 raft_consensus.cc:2229] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:44:43.553560 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 27856
I20260811 01:44:43.554013 28350 raft_consensus.cc:2258] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:43.554014 28351 raft_consensus.cc:2258] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:43.556401 28350 ts_tablet_manager.cc:1905] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:43.556401 28351 ts_tablet_manager.cc:1905] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:43.556846 27929 tablet_service.cc:1514] Processing DeleteTablet for tablet 528c0ed8e63642c9b071d1de01ab5fb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:44:43 UTC) from {username='slave'} at 127.0.0.1:48664
I20260811 01:44:43.563671 28350 ts_tablet_manager.cc:1918] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:44:43.563680 28351 ts_tablet_manager.cc:1918] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:44:43.563858 28351 log.cc:1196] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/528c0ed8e63642c9b071d1de01ab5fb2
I20260811 01:44:43.563915 28350 log.cc:1196] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/528c0ed8e63642c9b071d1de01ab5fb2
I20260811 01:44:43.564500 28351 ts_tablet_manager.cc:1939] T 528c0ed8e63642c9b071d1de01ab5fb2 P 64791d742cd2420b929e23514ca8785b: Deleting consensus metadata
I20260811 01:44:43.564500 28350 ts_tablet_manager.cc:1939] T 528c0ed8e63642c9b071d1de01ab5fb2 P 4008534ac65f4eee828e41b437e45c2b: Deleting consensus metadata
I20260811 01:44:43.566709 27771 catalog_manager.cc:4841] TS 64791d742cd2420b929e23514ca8785b (127.21.214.67:45347): tablet 528c0ed8e63642c9b071d1de01ab5fb2 (table test-table [id=7ab458624f2041e6a2c3b744874b4d1b]) successfully deleted
I20260811 01:44:43.566686 27774 catalog_manager.cc:4841] TS 4008534ac65f4eee828e41b437e45c2b (127.21.214.66:46729): tablet 528c0ed8e63642c9b071d1de01ab5fb2 (table test-table [id=7ab458624f2041e6a2c3b744874b4d1b]) successfully deleted
I20260811 01:44:43.568218 28353 tablet_replica.cc:330] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77: stopping tablet replica
I20260811 01:44:43.568710 28353 raft_consensus.cc:2229] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:44:43.569514 28353 raft_consensus.cc:2258] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:43.570079 28320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.555827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48682 (local address 127.21.214.65:45061)
0811 01:44:43.556018 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:43.556025 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:43.567478 (+ 11453us) server_negotiation.cc:407] Connection header received
0811 01:44:43.567744 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:43.567752 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:43.567835 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:43.567968 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:43.569301 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:43.569308 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:43.569313 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:43.569333 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:43.569378 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:43.569395 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:43.569570 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:43.569699 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 01:44:43.569765 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 01:44:43.570950 27994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:43.572260 28353 ts_tablet_manager.cc:1905] T 528c0ed8e63642c9b071d1de01ab5fb2 P f601565fb6874a218f02dd95489e8a77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:43.677515 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 27856
I20260811 01:44:43.702527 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 28004
W20260811 01:44:43.702543 27771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.214.65:45061 (error 108)
I20260811 01:44:43.704605 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.704032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:45061 (local address 127.0.0.1:48696)
0811 01:44:43.704252 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:43.704430 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:45061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 01:44:43.704877 27771 catalog_manager.cc:4584] TS f601565fb6874a218f02dd95489e8a77 (127.21.214.65:45061): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 528c0ed8e63642c9b071d1de01ab5fb2: Network error: Client connection negotiation failed: client connection to 127.21.214.65:45061: connect: Connection refused (error 111)
I20260811 01:44:43.706032 28319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.703657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34328 (local address 127.21.214.66:46729)
0811 01:44:43.703871 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:43.703879 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:43.703902 (+ 23us) server_negotiation.cc:407] Connection header received
0811 01:44:43.703976 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:43.703980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:43.704093 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:43.704204 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:43.705366 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:43.705372 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:43.705377 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:43.705403 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:43.705437 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:43.705449 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:43.705587 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:43.705682 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 01:44:43.705746 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 01:44:43.706929 28150 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:43.758556 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.758062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:45061 (local address 127.0.0.1:48700)
0811 01:44:43.758303 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:43.758395 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:45061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 01:44:43.759644 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.759282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:45061 (local address 127.0.0.1:48708)
0811 01:44:43.759432 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:43.759509 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:45061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 01:44:43.801683 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 28004
I20260811 01:44:43.820755 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.820097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:45061 (local address 127.0.0.1:48724)
0811 01:44:43.820456 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:43.820578 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:45061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208}
I20260811 01:44:43.821985 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.821596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:45061 (local address 127.0.0.1:48736)
0811 01:44:43.821752 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:43.821845 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:45061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 01:44:43.826002 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 28158
I20260811 01:44:43.829681 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.826802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37358 (local address 127.21.214.67:45347)
0811 01:44:43.826988 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:43.826996 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:43.827020 (+ 24us) server_negotiation.cc:407] Connection header received
0811 01:44:43.827286 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:43.827295 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:43.827463 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:43.827611 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:43.828808 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:43.828831 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:43.828840 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:43.828863 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:43.828911 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:43.828946 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:43.829103 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:43.829247 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 01:44:43.829321 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 01:44:43.830681 28297 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:43.902741 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.902195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:45061 (local address 127.0.0.1:48750)
0811 01:44:43.902454 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:43.902554 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:45061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260811 01:44:43.904096 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:43.903613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:45061 (local address 127.0.0.1:48766)
0811 01:44:43.903860 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:43.903958 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:45061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260811 01:44:43.920485 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 28158
I20260811 01:44:43.943197 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 27756
I20260811 01:44:43.944504 27823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:44.055728 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 27756
2026-08-11T01:44:44Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3001 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 01:44:44.104933 22361 mini_oidc.cc:152] Starting JWKS server
I20260811 01:44:44.108201 22361 webserver.cc:460] Webserver started at https://127.0.0.1:45149/ using document root <none> and password file <none>
I20260811 01:44:44.108592 22361 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 01:44:44.109396 22361 webserver.cc:460] Webserver started at http://127.0.0.1:41319/ using document root <none> and password file <none>
2026-08-11T01:44:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:44:44Z Disabled control of system clock
I20260811 01:44:44.146888 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:35423
--webserver_interface=127.21.214.126
--webserver_port=0
--builtin_ntp_servers=127.21.214.84:46217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:35423
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45149/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/testchainca.cert with env {}
W20260811 01:44:44.498378 28365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:44.498777 28365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:44.498942 28365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:44.506515 28365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:44:44.506676 28365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:44.506778 28365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:44.506891 28365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:44:44.506969 28365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 01:44:44.507061 28365 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 01:44:44.507139 28365 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45149/jwks.json
I20260811 01:44:44.518255 28365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:46217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:35423
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:35423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45149/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:44.519740 28365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:44.521173 28365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:44.534796 28371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:44.534866 28373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:44.534866 28370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:44.536075 28365 server_base.cc:1034] running on GCE node
I20260811 01:44:44.536943 28365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:44.538607 28365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:44.539868 28365 hybrid_clock.cc:648] HybridClock initialized: now 1786412684539837 us; error 66 us; skew 500 ppm
I20260811 01:44:44.540256 28365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:44:44.542862 28365 webserver.cc:460] Webserver started at http://127.21.214.126:38517/ using document root <none> and password file <none>
I20260811 01:44:44.543632 28365 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:44.543783 28365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:44.544116 28365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:44.547309 28365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "40a49ba3624b4bb3a9bb0899cf35fa56"
format_stamp: "Formatted at 2026-08-11 01:44:44 on dist-test-slave-9gcw"
server_key: "f32f5821935e6fd1d97b4e8f7a41549d"
server_key_iv: "c2280eea3577ddb7bfd27d843a5c70dc"
server_key_version: "encryptionkey@0"
I20260811 01:44:44.548166 28365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "40a49ba3624b4bb3a9bb0899cf35fa56"
format_stamp: "Formatted at 2026-08-11 01:44:44 on dist-test-slave-9gcw"
server_key: "f32f5821935e6fd1d97b4e8f7a41549d"
server_key_iv: "c2280eea3577ddb7bfd27d843a5c70dc"
server_key_version: "encryptionkey@0"
I20260811 01:44:44.554037 28365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 01:44:44.558406 28380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:44.560047 28365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260811 01:44:44.560304 28365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "40a49ba3624b4bb3a9bb0899cf35fa56"
format_stamp: "Formatted at 2026-08-11 01:44:44 on dist-test-slave-9gcw"
server_key: "f32f5821935e6fd1d97b4e8f7a41549d"
server_key_iv: "c2280eea3577ddb7bfd27d843a5c70dc"
server_key_version: "encryptionkey@0"
I20260811 01:44:44.560583 28365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:44.582295 28365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:44.583349 28365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:44.583735 28365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:44.604887 28365 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:35423
I20260811 01:44:44.604894 28442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:35423 every 8 connection(s)
I20260811 01:44:44.606570 28365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:44:44.610739 28443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:44.612205 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 28365
I20260811 01:44:44.612457 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:44:44.613345 22361 external_mini_cluster.cc:1428] Setting key d905720bb97445fbf35164a5506b7eb7
I20260811 01:44:44.626525 28443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56: Bootstrap starting.
I20260811 01:44:44.629436 28447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:44.615612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48502 (local address 127.21.214.126:35423)
0811 01:44:44.616841 (+ 1229us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:44.616855 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:44.616908 (+ 53us) server_negotiation.cc:407] Connection header received
0811 01:44:44.618795 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:44.618829 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:44.619372 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:44.619747 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:44.620435 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:44.621999 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:44.624111 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:44.624963 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:44.625204 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:44.626554 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:44.626562 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:44.626579 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:44.627283 (+ 704us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:44.627376 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:44.627399 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:44.627494 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:44.627920 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 01:44:44.628307 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":453,"threads_started":1}
I20260811 01:44:44.630290 28443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:44.631453 28443 log.cc:824] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:44.633785 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:44:44.635092 28443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56: No bootstrap required, opened a new log
I20260811 01:44:44.643340 28443 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a49ba3624b4bb3a9bb0899cf35fa56" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 35423 } }
I20260811 01:44:44.643775 28443 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:44.643850 28443 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a49ba3624b4bb3a9bb0899cf35fa56, State: Initialized, Role: FOLLOWER
I20260811 01:44:44.644431 28443 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a49ba3624b4bb3a9bb0899cf35fa56" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 35423 } }
I20260811 01:44:44.644641 28443 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:44:44.644752 28443 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:44:44.644881 28443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:44.647522 28443 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a49ba3624b4bb3a9bb0899cf35fa56" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 35423 } }
I20260811 01:44:44.648046 28443 leader_election.cc:302] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40a49ba3624b4bb3a9bb0899cf35fa56; no voters:
I20260811 01:44:44.648682 28443 leader_election.cc:288] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:44:44.648875 28449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:44.649715 28449 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56 [term 1 LEADER]: Becoming Leader. State: Replica: 40a49ba3624b4bb3a9bb0899cf35fa56, State: Running, Role: LEADER
I20260811 01:44:44.650383 28449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a49ba3624b4bb3a9bb0899cf35fa56" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 35423 } }
I20260811 01:44:44.651079 28443 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:44:44.654531 28452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40a49ba3624b4bb3a9bb0899cf35fa56. Latest consensus state: current_term: 1 leader_uuid: "40a49ba3624b4bb3a9bb0899cf35fa56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a49ba3624b4bb3a9bb0899cf35fa56" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 35423 } } }
I20260811 01:44:44.655412 28452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56 [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:44.654948 28451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40a49ba3624b4bb3a9bb0899cf35fa56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a49ba3624b4bb3a9bb0899cf35fa56" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 35423 } } }
I20260811 01:44:44.655984 28451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56 [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:44.657775 28458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:44:44.664062 28458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:44:44.674783 28458 catalog_manager.cc:1347] Generated new cluster ID: 71287aa7f4e64f459de32d6a573e30ee
I20260811 01:44:44.674939 28458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:44:44.696887 28458 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:44:44.698359 28458 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:44:44.714581 28458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40a49ba3624b4bb3a9bb0899cf35fa56: Generated new TSK 0
I20260811 01:44:44.715569 28458 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 01:44:44.761121 28476 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 01:44:44.761479 28476 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 01:44:44.763746 28447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:44.733935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48508 (local address 127.21.214.126:35423)
0811 01:44:44.734175 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:44.734183 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:44.734668 (+ 485us) server_negotiation.cc:407] Connection header received
0811 01:44:44.735044 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:44.735051 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:44.735152 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:44.735382 (+ 230us) server_negotiation.cc:226] Negotiated authn=JWT
0811 01:44:44.735993 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:44.737316 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:44.738957 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:44.739791 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:44.739991 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:44.740569 (+ 578us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 01:44:44.762934 (+ 22365us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45149/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 01:44:44.763375 (+ 441us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48508: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45149/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 01:44:44.763912 28447 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48508: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45149/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 01:44:44.765709 28475 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:44:44.733748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:35423 (local address 127.0.0.1:48508)
0811 01:44:44.734553 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:44.734571 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:44.734747 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:44.735400 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:44.735404 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:44.735413 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 01:44:44.735833 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:44.735844 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:44.737536 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:44.737541 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:44.738784 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:44.738798 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:44.740021 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:44.740025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:44.740165 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:44.740232 (+ 67us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 01:44:44.763296 (+ 23064us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45149/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 01:44:44.763596 (+ 300us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.214.126:35423: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45149/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":450,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.21.214.126:35423: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45149/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 01:44:44.767628 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 28365
I20260811 01:44:44.768980 28438 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:44.867895 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 28365
2026-08-11T01:44:44Z chronyd exiting
I20260811 01:44:44.924845 22361 test_util.cc:175] -----------------------------------------------
I20260811 01:44:44.925034 22361 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786412636131983-22361-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (825 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 01:44:44.928964 22361 mini_oidc.cc:152] Starting JWKS server
I20260811 01:44:44.931821 22361 webserver.cc:460] Webserver started at https://127.0.0.1:35065/ using document root <none> and password file <none>
I20260811 01:44:44.932189 22361 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 01:44:44.933081 22361 webserver.cc:460] Webserver started at http://127.0.0.1:37165/ using document root <none> and password file <none>
2026-08-11T01:44:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:44:44Z Disabled control of system clock
I20260811 01:44:44.970460 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:34771
--webserver_interface=127.21.214.126
--webserver_port=0
--builtin_ntp_servers=127.21.214.84:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:34771
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35065/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/testchainca.cert with env {}
W20260811 01:44:45.324005 28484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:45.324344 28484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:45.324529 28484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:45.332122 28484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:44:45.332249 28484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:45.332326 28484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:45.332422 28484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:44:45.332507 28484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 01:44:45.332605 28484 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 01:44:45.332643 28484 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35065/jwks.json
I20260811 01:44:45.343348 28484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:34771
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:34771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35065/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:45.344808 28484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:45.346223 28484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:45.359174 28494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:45.359251 28491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:45.359253 28490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:45.360263 28484 server_base.cc:1034] running on GCE node
I20260811 01:44:45.361107 28484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:45.362905 28484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:45.364152 28484 hybrid_clock.cc:648] HybridClock initialized: now 1786412685364105 us; error 69 us; skew 500 ppm
I20260811 01:44:45.364549 28484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:44:45.372699 28484 webserver.cc:460] Webserver started at http://127.21.214.126:41565/ using document root <none> and password file <none>
I20260811 01:44:45.373615 28484 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:45.373770 28484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:45.374153 28484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:45.377314 28484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "b9ae74d07f3a4fed93083f6311c71865"
format_stamp: "Formatted at 2026-08-11 01:44:45 on dist-test-slave-9gcw"
server_key: "824901100c193496d5b14fe47f6448fa"
server_key_iv: "175d19bbf39540c14fae413d889c3bf3"
server_key_version: "encryptionkey@0"
I20260811 01:44:45.378144 28484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "b9ae74d07f3a4fed93083f6311c71865"
format_stamp: "Formatted at 2026-08-11 01:44:45 on dist-test-slave-9gcw"
server_key: "824901100c193496d5b14fe47f6448fa"
server_key_iv: "175d19bbf39540c14fae413d889c3bf3"
server_key_version: "encryptionkey@0"
I20260811 01:44:45.383639 28484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 01:44:45.387862 28500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:45.389406 28484 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 01:44:45.389631 28484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "b9ae74d07f3a4fed93083f6311c71865"
format_stamp: "Formatted at 2026-08-11 01:44:45 on dist-test-slave-9gcw"
server_key: "824901100c193496d5b14fe47f6448fa"
server_key_iv: "175d19bbf39540c14fae413d889c3bf3"
server_key_version: "encryptionkey@0"
I20260811 01:44:45.389967 28484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:45.426067 28484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:45.427062 28484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:45.427441 28484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:45.448254 28484 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:34771
I20260811 01:44:45.448276 28560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:34771 every 8 connection(s)
I20260811 01:44:45.449709 28484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:44:45.453821 28561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:45.457033 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 28484
I20260811 01:44:45.457288 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:44:45.457996 22361 external_mini_cluster.cc:1428] Setting key a8632b3a26331ebcff9b65ce554e62d0
I20260811 01:44:45.468492 28561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865: Bootstrap starting.
I20260811 01:44:45.472860 28561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:45.474387 28561 log.cc:824] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:45.474676 28564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:45.460130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42584 (local address 127.21.214.126:34771)
0811 01:44:45.461393 (+ 1263us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:45.461412 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:45.461467 (+ 55us) server_negotiation.cc:407] Connection header received
0811 01:44:45.463558 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:45.463595 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:45.464313 (+ 718us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:45.464864 (+ 551us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:45.465518 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:45.467534 (+ 2016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:45.469006 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:45.470078 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:45.470291 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:45.471511 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:45.471523 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:45.471542 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:45.472299 (+ 757us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:45.472429 (+ 130us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:45.472459 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:45.472585 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:45.472845 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 01:44:45.473248 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":348,"threads_started":1}
I20260811 01:44:45.479218 28561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865: No bootstrap required, opened a new log
I20260811 01:44:45.479362 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:44:45.487648 28561 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae74d07f3a4fed93083f6311c71865" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34771 } }
I20260811 01:44:45.488078 28561 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:45.488206 28561 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9ae74d07f3a4fed93083f6311c71865, State: Initialized, Role: FOLLOWER
I20260811 01:44:45.488910 28561 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae74d07f3a4fed93083f6311c71865" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34771 } }
I20260811 01:44:45.489228 28561 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:44:45.489392 28561 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:44:45.489573 28561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:45.492210 28561 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae74d07f3a4fed93083f6311c71865" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34771 } }
I20260811 01:44:45.492715 28561 leader_election.cc:302] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9ae74d07f3a4fed93083f6311c71865; no voters:
I20260811 01:44:45.493360 28561 leader_election.cc:288] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:44:45.493460 28566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:45.494374 28566 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865 [term 1 LEADER]: Becoming Leader. State: Replica: b9ae74d07f3a4fed93083f6311c71865, State: Running, Role: LEADER
I20260811 01:44:45.495062 28566 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae74d07f3a4fed93083f6311c71865" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34771 } }
I20260811 01:44:45.496029 28561 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:44:45.497874 28568 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9ae74d07f3a4fed93083f6311c71865. Latest consensus state: current_term: 1 leader_uuid: "b9ae74d07f3a4fed93083f6311c71865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae74d07f3a4fed93083f6311c71865" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34771 } } }
I20260811 01:44:45.498461 28568 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865 [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:45.498442 28567 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9ae74d07f3a4fed93083f6311c71865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae74d07f3a4fed93083f6311c71865" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34771 } } }
I20260811 01:44:45.499075 28567 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865 [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:45.500675 28572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:44:45.506389 28572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:44:45.516387 28572 catalog_manager.cc:1347] Generated new cluster ID: 2eb759756e2b497eb2b318a36b819be3
I20260811 01:44:45.516517 28572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:44:45.530090 28572 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:44:45.531418 28572 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:44:45.541030 28572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9ae74d07f3a4fed93083f6311c71865: Generated new TSK 0
I20260811 01:44:45.541941 28572 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 01:44:45.573400 28596 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 01:44:45.573544 28596 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 01:44:45.574258 28564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:45.554766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42598 (local address 127.21.214.126:34771)
0811 01:44:45.555007 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:45.555014 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:45.555297 (+ 283us) server_negotiation.cc:407] Connection header received
0811 01:44:45.555590 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:45.555595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:45.555668 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:45.555838 (+ 170us) server_negotiation.cc:226] Negotiated authn=JWT
0811 01:44:45.556359 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:45.557111 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:45.558570 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:45.559067 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:45.559219 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:45.559745 (+ 526us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 01:44:45.573604 (+ 13859us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35065/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 01:44:45.574032 (+ 428us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42598: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35065/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 01:44:45.574416 28564 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42598: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35065/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 01:44:45.574659 28595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:44:45.554573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:34771 (local address 127.0.0.1:42598)
0811 01:44:45.555179 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:45.555196 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:45.555388 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:45.555856 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:45.555861 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:45.555872 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0811 01:44:45.556167 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:45.556176 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:45.557309 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:45.557322 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:45.558415 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:45.558427 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:45.559286 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:45.559295 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:45.559450 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:45.559483 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 01:44:45.573924 (+ 14441us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35065/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 01:44:45.574252 (+ 328us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.214.126:34771: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35065/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":335,"threads_started":1}
I20260811 01:44:45.576047 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 28484
I20260811 01:44:45.577705 28556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:45.671988 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 28484
2026-08-11T01:44:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (804 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 01:44:45.733925 22361 mini_oidc.cc:152] Starting JWKS server
I20260811 01:44:45.735940 22361 webserver.cc:460] Webserver started at https://127.0.0.1:41841/ using document root <none> and password file <none>
I20260811 01:44:45.736228 22361 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 01:44:45.736910 22361 webserver.cc:460] Webserver started at http://127.0.0.1:44951/ using document root <none> and password file <none>
2026-08-11T01:44:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:44:45Z Disabled control of system clock
I20260811 01:44:45.772894 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:44655
--webserver_interface=127.21.214.126
--webserver_port=0
--builtin_ntp_servers=127.21.214.84:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:44655
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41841/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 01:44:46.119355 28603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:46.119688 28603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:46.119807 28603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:46.127409 28603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:44:46.127534 28603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:46.127599 28603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:46.127700 28603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:44:46.127763 28603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 01:44:46.127857 28603 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 01:44:46.127935 28603 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41841/jwks.json
I20260811 01:44:46.137581 28603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:44655
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:44655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41841/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:46.139214 28603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:46.141122 28603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:46.153931 28608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:46.154003 28612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:46.154027 28610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:46.155011 28603 server_base.cc:1034] running on GCE node
I20260811 01:44:46.156039 28603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:46.157676 28603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:46.158946 28603 hybrid_clock.cc:648] HybridClock initialized: now 1786412686158878 us; error 56 us; skew 500 ppm
I20260811 01:44:46.159411 28603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:44:46.162283 28603 webserver.cc:460] Webserver started at http://127.21.214.126:35417/ using document root <none> and password file <none>
I20260811 01:44:46.163092 28603 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:46.163244 28603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:46.163647 28603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:46.167027 28603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "565c16196f04475391a7697bd4f042b0"
format_stamp: "Formatted at 2026-08-11 01:44:46 on dist-test-slave-9gcw"
server_key: "94df412e129883bb6f983cf8a45d1ad3"
server_key_iv: "6eaf3e7a03fcf0636442b57ae9aacf6a"
server_key_version: "encryptionkey@0"
I20260811 01:44:46.167948 28603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "565c16196f04475391a7697bd4f042b0"
format_stamp: "Formatted at 2026-08-11 01:44:46 on dist-test-slave-9gcw"
server_key: "94df412e129883bb6f983cf8a45d1ad3"
server_key_iv: "6eaf3e7a03fcf0636442b57ae9aacf6a"
server_key_version: "encryptionkey@0"
I20260811 01:44:46.173979 28603 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 01:44:46.178007 28619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:46.179749 28603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:44:46.180001 28603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "565c16196f04475391a7697bd4f042b0"
format_stamp: "Formatted at 2026-08-11 01:44:46 on dist-test-slave-9gcw"
server_key: "94df412e129883bb6f983cf8a45d1ad3"
server_key_iv: "6eaf3e7a03fcf0636442b57ae9aacf6a"
server_key_version: "encryptionkey@0"
I20260811 01:44:46.180287 28603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:46.203712 28603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:46.204788 28603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:46.205206 28603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:46.227411 28603 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:44655
I20260811 01:44:46.227433 28682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:44655 every 8 connection(s)
I20260811 01:44:46.228997 28603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:44:46.233129 28683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:46.237682 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 28603
I20260811 01:44:46.237949 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:44:46.238689 22361 external_mini_cluster.cc:1428] Setting key bef56b0438b2a99145b216d28e7730f9
I20260811 01:44:46.246781 28683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0: Bootstrap starting.
I20260811 01:44:46.250798 28683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:46.252206 28683 log.cc:824] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:46.255981 28683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0: No bootstrap required, opened a new log
I20260811 01:44:46.256214 28686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:46.241019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53016 (local address 127.21.214.126:44655)
0811 01:44:46.242307 (+ 1288us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:46.242324 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:46.242385 (+ 61us) server_negotiation.cc:407] Connection header received
0811 01:44:46.244971 (+ 2586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:46.245003 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:46.245671 (+ 668us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:46.246171 (+ 500us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:46.246766 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:46.248868 (+ 2102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:46.250441 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:46.251591 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:46.251807 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:46.253276 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:46.253287 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:46.253303 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:46.254006 (+ 703us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:46.254118 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:46.254147 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:46.254266 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:46.254505 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 01:44:46.254870 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":363,"threads_started":1}
I20260811 01:44:46.260427 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:44:46.264981 28683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565c16196f04475391a7697bd4f042b0" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44655 } }
I20260811 01:44:46.265503 28683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:46.265621 28683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565c16196f04475391a7697bd4f042b0, State: Initialized, Role: FOLLOWER
I20260811 01:44:46.266245 28683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565c16196f04475391a7697bd4f042b0" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44655 } }
I20260811 01:44:46.266525 28683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:44:46.266726 28683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:44:46.266924 28683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:46.269322 28683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565c16196f04475391a7697bd4f042b0" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44655 } }
I20260811 01:44:46.269905 28683 leader_election.cc:302] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 565c16196f04475391a7697bd4f042b0; no voters:
I20260811 01:44:46.270632 28683 leader_election.cc:288] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:44:46.270807 28688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:46.271770 28688 raft_consensus.cc:686] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0 [term 1 LEADER]: Becoming Leader. State: Replica: 565c16196f04475391a7697bd4f042b0, State: Running, Role: LEADER
I20260811 01:44:46.272336 28688 consensus_queue.cc:237] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565c16196f04475391a7697bd4f042b0" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44655 } }
I20260811 01:44:46.273075 28683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:44:46.276011 28689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "565c16196f04475391a7697bd4f042b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565c16196f04475391a7697bd4f042b0" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44655 } } }
I20260811 01:44:46.276518 28689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0 [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:46.276883 28690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 565c16196f04475391a7697bd4f042b0. Latest consensus state: current_term: 1 leader_uuid: "565c16196f04475391a7697bd4f042b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "565c16196f04475391a7697bd4f042b0" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44655 } } }
I20260811 01:44:46.277508 28690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0 [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:46.285835 28700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:44:46.291512 28700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:44:46.302554 28700 catalog_manager.cc:1347] Generated new cluster ID: 04088628cec648db857fa2d7fd96b18d
I20260811 01:44:46.302686 28700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:44:46.316613 28700 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:44:46.318670 28700 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:44:46.336390 28700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 565c16196f04475391a7697bd4f042b0: Generated new TSK 0
I20260811 01:44:46.337728 28700 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:44:46.362660 28686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:46.353634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53030 (local address 127.21.214.126:44655)
0811 01:44:46.353857 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:46.353865 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:46.354368 (+ 503us) server_negotiation.cc:407] Connection header received
0811 01:44:46.354591 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:46.354597 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:46.354695 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:46.354913 (+ 218us) server_negotiation.cc:226] Negotiated authn=JWT
0811 01:44:46.355385 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:46.356579 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:46.358083 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:46.358796 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:46.358930 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:46.359456 (+ 526us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 01:44:46.361948 (+ 2492us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41841/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 01:44:46.362333 (+ 385us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53030: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41841/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 01:44:46.362798 28686 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53030: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41841/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 01:44:46.364351 28721 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:44:46.353600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:44655 (local address 127.0.0.1:53030)
0811 01:44:46.354256 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:46.354274 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:46.354403 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:46.354939 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:46.354943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:46.354952 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 01:44:46.355219 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:46.355226 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:46.356751 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:46.356755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:46.357933 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:46.357946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:46.358996 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:46.359001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:46.359138 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:46.359171 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 01:44:46.362179 (+ 3008us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41841/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 01:44:46.363919 (+ 1740us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.214.126:44655: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41841/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":1420,"thread_start_us":366,"threads_started":1}
I20260811 01:44:46.365525 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 28603
I20260811 01:44:46.367416 28678 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 01:44:46.375413 28722 webserver.cc:533] Webserver: sslize failed:
E20260811 01:44:46.375509 28722 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 01:44:46.461480 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 28603
2026-08-11T01:44:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (787 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T01:44:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:44:46Z Disabled control of system clock
I20260811 01:44:46.557394 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:34727
--webserver_interface=127.21.214.126
--webserver_port=0
--builtin_ntp_servers=127.21.214.84:37739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:34727
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/testchainca.cert with env {}
W20260811 01:44:46.907351 28728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:46.907691 28728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:46.907871 28728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:46.915537 28728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:44:46.915669 28728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:46.915709 28728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:46.915793 28728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:44:46.915880 28728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 01:44:46.915994 28728 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 01:44:46.916054 28728 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 01:44:46.926713 28728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:37739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:34727
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:34727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:46.928654 28728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:46.930574 28728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:46.943193 28733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:46.943140 28736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:46.943213 28734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:46.944093 28728 server_base.cc:1034] running on GCE node
I20260811 01:44:46.944893 28728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:46.946511 28728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:46.947798 28728 hybrid_clock.cc:648] HybridClock initialized: now 1786412686947745 us; error 48 us; skew 500 ppm
I20260811 01:44:46.948228 28728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:44:46.951095 28728 webserver.cc:460] Webserver started at http://127.21.214.126:46719/ using document root <none> and password file <none>
I20260811 01:44:46.951848 28728 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:46.951942 28728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:46.952308 28728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:46.955461 28728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "44bd9d20ac8d4b09b720185deebb514d"
format_stamp: "Formatted at 2026-08-11 01:44:46 on dist-test-slave-9gcw"
server_key: "0af534ad68921d54d29b252e9b2d6999"
server_key_iv: "4c8e75e86849ea4dffcd44e4bb3d0e69"
server_key_version: "encryptionkey@0"
I20260811 01:44:46.956321 28728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "44bd9d20ac8d4b09b720185deebb514d"
format_stamp: "Formatted at 2026-08-11 01:44:46 on dist-test-slave-9gcw"
server_key: "0af534ad68921d54d29b252e9b2d6999"
server_key_iv: "4c8e75e86849ea4dffcd44e4bb3d0e69"
server_key_version: "encryptionkey@0"
I20260811 01:44:46.962188 28728 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260811 01:44:46.966050 28743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:46.967566 28728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:44:46.967852 28728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "44bd9d20ac8d4b09b720185deebb514d"
format_stamp: "Formatted at 2026-08-11 01:44:46 on dist-test-slave-9gcw"
server_key: "0af534ad68921d54d29b252e9b2d6999"
server_key_iv: "4c8e75e86849ea4dffcd44e4bb3d0e69"
server_key_version: "encryptionkey@0"
I20260811 01:44:46.968127 28728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:46.989287 28728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:46.990244 28728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:46.990579 28728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:47.011603 28728 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:34727
I20260811 01:44:47.011623 28799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:34727 every 8 connection(s)
I20260811 01:44:47.013324 28728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:44:47.017378 28800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:47.022907 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 28728
I20260811 01:44:47.023147 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:44:47.024012 22361 external_mini_cluster.cc:1428] Setting key 20df1e8742b8377ef8b10f04b10743b3
I20260811 01:44:47.028447 28800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d: Bootstrap starting.
I20260811 01:44:47.031955 28800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:47.033071 28800 log.cc:824] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:47.036064 28800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d: No bootstrap required, opened a new log
I20260811 01:44:47.040330 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:47.026252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57046 (local address 127.21.214.126:34727)
0811 01:44:47.027471 (+ 1219us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:47.027483 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:47.027534 (+ 51us) server_negotiation.cc:407] Connection header received
0811 01:44:47.029657 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:47.029682 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:47.030249 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:47.030651 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:47.031337 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:47.033080 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:47.034720 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:47.035578 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:47.035834 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:47.037382 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:47.037392 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:47.037409 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:44:47.038103 (+ 694us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:47.038210 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:44:47.038236 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:47.038423 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:47.038777 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 01:44:47.039139 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":451,"threads_started":1}
I20260811 01:44:47.045101 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:44:47.045059 28800 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44bd9d20ac8d4b09b720185deebb514d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34727 } }
I20260811 01:44:47.045529 28800 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:47.045637 28800 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44bd9d20ac8d4b09b720185deebb514d, State: Initialized, Role: FOLLOWER
I20260811 01:44:47.046252 28800 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44bd9d20ac8d4b09b720185deebb514d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34727 } }
I20260811 01:44:47.046502 28800 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:44:47.046648 28800 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:44:47.046837 28800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:47.049443 28800 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44bd9d20ac8d4b09b720185deebb514d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34727 } }
I20260811 01:44:47.049957 28800 leader_election.cc:302] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44bd9d20ac8d4b09b720185deebb514d; no voters:
I20260811 01:44:47.050621 28800 leader_election.cc:288] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:44:47.050853 28806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:47.052112 28806 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d [term 1 LEADER]: Becoming Leader. State: Replica: 44bd9d20ac8d4b09b720185deebb514d, State: Running, Role: LEADER
I20260811 01:44:47.052744 28806 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44bd9d20ac8d4b09b720185deebb514d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34727 } }
I20260811 01:44:47.053014 28800 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:44:47.056334 28809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44bd9d20ac8d4b09b720185deebb514d. Latest consensus state: current_term: 1 leader_uuid: "44bd9d20ac8d4b09b720185deebb514d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44bd9d20ac8d4b09b720185deebb514d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34727 } } }
I20260811 01:44:47.056841 28809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:47.056322 28808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44bd9d20ac8d4b09b720185deebb514d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44bd9d20ac8d4b09b720185deebb514d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34727 } } }
I20260811 01:44:47.057505 28808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:47.058950 28816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:44:47.065366 28816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:44:47.074667 28816 catalog_manager.cc:1347] Generated new cluster ID: 553943a95c844a6b9915ed5abf4452f9
I20260811 01:44:47.074813 28816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:44:47.116917 28816 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:44:47.118876 28816 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:44:47.136991 28816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44bd9d20ac8d4b09b720185deebb514d: Generated new TSK 0
I20260811 01:44:47.138474 28816 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:44:47.612015 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:47.164389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57062 (local address 127.21.214.126:34727)
0811 01:44:47.164594 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:47.164601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:47.164939 (+ 338us) server_negotiation.cc:407] Connection header received
0811 01:44:47.165214 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:47.165220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:47.165314 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:47.165511 (+ 197us) server_negotiation.cc:226] Negotiated authn=JWT
0811 01:44:47.165943 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:47.167132 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:47.168623 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:47.169365 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:47.169495 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:47.169985 (+ 490us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 01:44:47.611402 (+441417us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 01:44:47.611761 (+ 359us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57062: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 01:44:47.612125 28803 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57062: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 01:44:47.612516 28831 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:44:47.164202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:34727 (local address 127.0.0.1:57062)
0811 01:44:47.164825 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:47.164842 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:47.164977 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:47.165522 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:47.165526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:47.165535 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 01:44:47.165793 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:47.165801 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:47.167349 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:47.167352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:47.168471 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:47.168483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:47.169527 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:47.169531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:47.169668 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:47.169702 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 01:44:47.611748 (+442046us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 01:44:47.612022 (+ 274us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.214.126:34727: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":329,"threads_started":1}
I20260811 01:44:47.614822 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 28728
I20260811 01:44:47.616093 28792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:47.720666 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 28728
2026-08-11T01:44:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1242 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (442 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (442 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:44:48 dist-test-slave-9gcw krb5kdc[28848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:44:48 dist-test-slave-9gcw krb5kdc[28848](info): set up 2 sockets
Aug 11 01:44:48 dist-test-slave-9gcw krb5kdc[28848](info): commencing operation
krb5kdc: starting...
W20260811 01:44:49.886410 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.002s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:44:50 dist-test-slave-9gcw krb5kdc[28848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:44:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:44:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:44:50.161631 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:34375
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:39935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:34375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:50.552209 28861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:50.552538 28861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:50.552731 28861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:50.560200 28861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:44:50.560324 28861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:50.560374 28861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:50.560509 28861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:44:50.560626 28861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:44:50.570955 28861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:39935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:34375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:34375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:50.572489 28861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:50.573904 28861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:50.588080 28866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:50.587860 28867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:50.588248 28869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:50.589538 28861 server_base.cc:1034] running on GCE node
I20260811 01:44:50.590562 28861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:50.592343 28861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:50.593612 28861 hybrid_clock.cc:648] HybridClock initialized: now 1786412690593543 us; error 86 us; skew 500 ppm
I20260811 01:44:50.594061 28861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:50 dist-test-slave-9gcw krb5kdc[28848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:50.601965 28861 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:44:50.603742 28861 webserver.cc:460] Webserver started at http://127.21.214.126:46017/ using document root <none> and password file <none>
I20260811 01:44:50.604557 28861 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:50.604722 28861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:50.605082 28861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:50.608306 28861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "1c9957ad44d1408e97fe7f1ed44c846f"
format_stamp: "Formatted at 2026-08-11 01:44:50 on dist-test-slave-9gcw"
server_key: "cd8edef7a80c4b5e93110f2ed8753018"
server_key_iv: "397c0b3dfdcf8e50c2b1d8fec1334ec3"
server_key_version: "encryptionkey@0"
I20260811 01:44:50.609207 28861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "1c9957ad44d1408e97fe7f1ed44c846f"
format_stamp: "Formatted at 2026-08-11 01:44:50 on dist-test-slave-9gcw"
server_key: "cd8edef7a80c4b5e93110f2ed8753018"
server_key_iv: "397c0b3dfdcf8e50c2b1d8fec1334ec3"
server_key_version: "encryptionkey@0"
I20260811 01:44:50.615051 28861 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 01:44:50.619200 28877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:50.620862 28861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 01:44:50.621260 28861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "1c9957ad44d1408e97fe7f1ed44c846f"
format_stamp: "Formatted at 2026-08-11 01:44:50 on dist-test-slave-9gcw"
server_key: "cd8edef7a80c4b5e93110f2ed8753018"
server_key_iv: "397c0b3dfdcf8e50c2b1d8fec1334ec3"
server_key_version: "encryptionkey@0"
I20260811 01:44:50.621524 28861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:50.648095 28861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:50.652911 28861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:50.653276 28861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:50.674965 28861 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:34375
I20260811 01:44:50.675007 28936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:34375 every 8 connection(s)
I20260811 01:44:50.676561 28861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:44:50.680437 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 28861
I20260811 01:44:50.680704 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:44:50.681221 28938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:50.681511 22361 external_mini_cluster.cc:1428] Setting key e7a4f4dd82266174b93b2504f25f1a32
Aug 11 01:44:50 dist-test-slave-9gcw krb5kdc[28848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:50.699697 28938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f: Bootstrap starting.
I20260811 01:44:50.704188 28938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:50.705583 28938 log.cc:824] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:50.708868 28941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:50.683888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45144 (local address 127.21.214.126:34375)
0811 01:44:50.684909 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:50.684933 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:50.685007 (+ 74us) server_negotiation.cc:407] Connection header received
0811 01:44:50.687711 (+ 2704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:50.687750 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:50.688412 (+ 662us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:50.688985 (+ 573us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:50.690477 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:50.692921 (+ 2444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:50.694351 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:50.695725 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:50.695994 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:50.700810 (+ 4816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:50.700819 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:50.700836 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:50.700877 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:50.704332 (+ 3455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:50.705270 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:50.705278 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:50.705285 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:50.705466 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:50.705917 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:50.705921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:50.705923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:50.706517 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:50.706869 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:50.707373 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 01:44:50.707829 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":401,"threads_started":1}
I20260811 01:44:50.709560 28938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f: No bootstrap required, opened a new log
I20260811 01:44:50.719417 28938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c9957ad44d1408e97fe7f1ed44c846f" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34375 } }
I20260811 01:44:50.719822 28938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:50.719902 28938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c9957ad44d1408e97fe7f1ed44c846f, State: Initialized, Role: FOLLOWER
I20260811 01:44:50.720427 28938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c9957ad44d1408e97fe7f1ed44c846f" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34375 } }
I20260811 01:44:50.720715 28938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:44:50.720856 28938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:44:50.720980 28938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:50.723582 28938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c9957ad44d1408e97fe7f1ed44c846f" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34375 } }
I20260811 01:44:50.724079 28938 leader_election.cc:302] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c9957ad44d1408e97fe7f1ed44c846f; no voters:
I20260811 01:44:50.724680 28938 leader_election.cc:288] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:44:50.724820 28944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:50.725565 28944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f [term 1 LEADER]: Becoming Leader. State: Replica: 1c9957ad44d1408e97fe7f1ed44c846f, State: Running, Role: LEADER
I20260811 01:44:50.726217 28944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c9957ad44d1408e97fe7f1ed44c846f" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34375 } }
I20260811 01:44:50.727162 28938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:44:50.729815 28945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c9957ad44d1408e97fe7f1ed44c846f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c9957ad44d1408e97fe7f1ed44c846f" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34375 } } }
I20260811 01:44:50.730064 28946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c9957ad44d1408e97fe7f1ed44c846f. Latest consensus state: current_term: 1 leader_uuid: "1c9957ad44d1408e97fe7f1ed44c846f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c9957ad44d1408e97fe7f1ed44c846f" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34375 } } }
I20260811 01:44:50.730417 28946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:50.730422 28945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:50.732151 28959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:44:50.737691 28959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:44:50.747965 28959 catalog_manager.cc:1347] Generated new cluster ID: 9ac9f3ac442846c2bd5c770b74b4d820
I20260811 01:44:50.748193 28959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:44:50.763850 28959 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:44:50.765379 28959 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:44:50.776075 28959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c9957ad44d1408e97fe7f1ed44c846f: Generated new TSK 0
I20260811 01:44:50.777035 28959 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
I20260811 01:44:50.846518 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:34375
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:39935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:51.205058 28971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:51.205451 28971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:51.205554 28971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:51.213342 28971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:51.213465 28971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:51.213580 28971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:44:51.224036 28971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:39935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:34375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:51.225498 28971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:51.226899 28971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:51.241650 28985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:51.241700 28983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:51.241765 28980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:51.242959 28971 server_base.cc:1034] running on GCE node
I20260811 01:44:51.243639 28971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:51.244720 28971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:51.245949 28971 hybrid_clock.cc:648] HybridClock initialized: now 1786412691245909 us; error 74 us; skew 500 ppm
I20260811 01:44:51.246301 28971 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:51 dist-test-slave-9gcw krb5kdc[28848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412691, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:51.249948 28971 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:44:51.251577 28971 webserver.cc:460] Webserver started at http://127.21.214.65:42771/ using document root <none> and password file <none>
I20260811 01:44:51.252341 28971 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:51.252501 28971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:51.252858 28971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:51.256110 28971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "e64bd6dd3a7f40bcbdf29b67fc426710"
format_stamp: "Formatted at 2026-08-11 01:44:51 on dist-test-slave-9gcw"
server_key: "b83a1fe6041aa12966590c7418de64e9"
server_key_iv: "6a9e41e1385df6276e7ce56205cc2a53"
server_key_version: "encryptionkey@0"
I20260811 01:44:51.256923 28971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "e64bd6dd3a7f40bcbdf29b67fc426710"
format_stamp: "Formatted at 2026-08-11 01:44:51 on dist-test-slave-9gcw"
server_key: "b83a1fe6041aa12966590c7418de64e9"
server_key_iv: "6a9e41e1385df6276e7ce56205cc2a53"
server_key_version: "encryptionkey@0"
I20260811 01:44:51.262363 28971 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 01:44:51.266324 28992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:51.268003 28971 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 01:44:51.268229 28971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "e64bd6dd3a7f40bcbdf29b67fc426710"
format_stamp: "Formatted at 2026-08-11 01:44:51 on dist-test-slave-9gcw"
server_key: "b83a1fe6041aa12966590c7418de64e9"
server_key_iv: "6a9e41e1385df6276e7ce56205cc2a53"
server_key_version: "encryptionkey@0"
I20260811 01:44:51.268488 28971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:51.289048 28971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:51.294225 28971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:51.294579 28971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:51.295722 28971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:51.297726 28971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:51.297844 28971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:51.298012 28971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:51.298087 28971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:51.334751 28971 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:38403
I20260811 01:44:51.334846 29119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:38403 every 8 connection(s)
I20260811 01:44:51.336146 28971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:44:51.344873 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 28971
I20260811 01:44:51.345127 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:44:51.346180 22361 external_mini_cluster.cc:1428] Setting key 921035cc2e308b034c73265e32f44ec3
Aug 11 01:44:51 dist-test-slave-9gcw krb5kdc[28848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412691, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:51.364612 29123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:51.339137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:42719 (local address 127.21.214.126:34375)
0811 01:44:51.339639 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:51.339650 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:51.340572 (+ 922us) server_negotiation.cc:407] Connection header received
0811 01:44:51.342565 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:51.342570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:51.342670 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:51.342792 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:51.344376 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:51.345236 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:51.348100 (+ 2864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:51.348922 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:51.349077 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:51.354918 (+ 5841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:51.354924 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:51.354928 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:51.354963 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:51.359087 (+ 4124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:51.360489 (+ 1402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:51.360495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:51.360497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:51.360705 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:51.361412 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:51.361422 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:51.361427 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:51.362189 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:51.362425 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:51.363727 (+ 1302us) server_negotiation.cc:299] Negotiation successful
0811 01:44:51.364061 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":257,"threads_started":1}
I20260811 01:44:51.367835 29124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:51.339583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:34375 (local address 127.21.214.65:42719)
0811 01:44:51.340405 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:51.340446 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:51.342270 (+ 1824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:51.343051 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:51.343059 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:51.343593 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:51.344188 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:51.344207 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:51.345420 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:51.345424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:51.347909 (+ 2485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:51.347926 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:51.349242 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:51.349253 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:51.349618 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:51.350544 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:51.350593 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:51.354378 (+ 3785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:51.359292 (+ 4914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:51.359302 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:51.359323 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:51.360126 (+ 803us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:51.360871 (+ 745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:51.360879 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:51.360883 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:51.361243 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:51.362522 (+ 1279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:51.362535 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:51.363125 (+ 590us) client_negotiation.cc:769] Sending connection context
0811 01:44:51.364932 (+ 1807us) client_negotiation.cc:240] Negotiation successful
0811 01:44:51.365799 (+ 867us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":349,"threads_started":1}
I20260811 01:44:51.370182 29120 heartbeater.cc:347] Connected to a master server at 127.21.214.126:34375
I20260811 01:44:51.370647 29120 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:51.371775 29120 heartbeater.cc:511] Master 127.21.214.126:34375 requested a full tablet report, sending...
I20260811 01:44:51.374935 28895 ts_manager.cc:194] Registered new tserver with Master: e64bd6dd3a7f40bcbdf29b67fc426710 (127.21.214.65:38403)
I20260811 01:44:51.377429 28895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:42719
WARNING: no policy specified for kudu/127.21.214.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
I20260811 01:44:51.425395 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:34375
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:39935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:51.785060 29127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:51.785524 29127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:51.785753 29127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:51.793500 29127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:51.793681 29127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:51.793867 29127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:44:51.804299 29127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:39935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:34375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:51.805799 29127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:51.807192 29127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:51.821555 29133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:51.821507 29132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:51.821594 29135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:51.822666 29127 server_base.cc:1034] running on GCE node
I20260811 01:44:51.823367 29127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:51.824420 29127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:51.825688 29127 hybrid_clock.cc:648] HybridClock initialized: now 1786412691825623 us; error 51 us; skew 500 ppm
I20260811 01:44:51.826134 29127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:51 dist-test-slave-9gcw krb5kdc[28848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412691, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:51.829799 29127 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:44:51.831539 29127 webserver.cc:460] Webserver started at http://127.21.214.66:33749/ using document root <none> and password file <none>
I20260811 01:44:51.832289 29127 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:51.832450 29127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:51.832831 29127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:51.836218 29127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "1f348f0d3efc4888987bc8dca670f075"
format_stamp: "Formatted at 2026-08-11 01:44:51 on dist-test-slave-9gcw"
server_key: "0240cc4e666f352fba0a1a85cea2a93a"
server_key_iv: "5f43e1038cd47c59494b6b873764e19b"
server_key_version: "encryptionkey@0"
I20260811 01:44:51.837124 29127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "1f348f0d3efc4888987bc8dca670f075"
format_stamp: "Formatted at 2026-08-11 01:44:51 on dist-test-slave-9gcw"
server_key: "0240cc4e666f352fba0a1a85cea2a93a"
server_key_iv: "5f43e1038cd47c59494b6b873764e19b"
server_key_version: "encryptionkey@0"
I20260811 01:44:51.842782 29127 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 01:44:51.846441 29144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:51.847921 29127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 01:44:51.848129 29127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "1f348f0d3efc4888987bc8dca670f075"
format_stamp: "Formatted at 2026-08-11 01:44:51 on dist-test-slave-9gcw"
server_key: "0240cc4e666f352fba0a1a85cea2a93a"
server_key_iv: "5f43e1038cd47c59494b6b873764e19b"
server_key_version: "encryptionkey@0"
I20260811 01:44:51.848361 29127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:51.879155 29127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:51.883893 29127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:51.884214 29127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:51.885366 29127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:51.886998 29127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:51.887108 29127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:51.887267 29127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:51.887346 29127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:51.925912 29127 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:42851
I20260811 01:44:51.926005 29272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:42851 every 8 connection(s)
I20260811 01:44:51.927397 29127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:44:51.933977 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 29127
I20260811 01:44:51.934208 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:44:51.934744 22361 external_mini_cluster.cc:1428] Setting key 286ae6644c451f05902030afe4888310
Aug 11 01:44:51 dist-test-slave-9gcw krb5kdc[28848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412691, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:51.951341 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:51.930376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:45007 (local address 127.21.214.126:34375)
0811 01:44:51.930874 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:51.930880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:51.932048 (+ 1168us) server_negotiation.cc:407] Connection header received
0811 01:44:51.934104 (+ 2056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:51.934110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:51.934212 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:51.934342 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:51.936091 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:51.937228 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:51.938899 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:51.939807 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:51.939976 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:51.943991 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:51.944002 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:51.944011 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:51.944063 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:51.947582 (+ 3519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:51.948614 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:51.948618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:51.948621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:51.948779 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:51.949371 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:51.949375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:51.949377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:51.949677 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:51.949828 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:51.950807 (+ 979us) server_negotiation.cc:299] Negotiation successful
0811 01:44:51.951002 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":251,"threads_started":1}
I20260811 01:44:51.953007 29277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:51.931033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:34375 (local address 127.21.214.66:45007)
0811 01:44:51.931872 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:51.931912 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:51.933808 (+ 1896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:51.934603 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:51.934612 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:51.935205 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:51.935900 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:51.935926 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:51.937411 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:51.937415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:51.938710 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:51.938722 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:51.940027 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:51.940031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:51.940251 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:51.941022 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:51.941044 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:51.943709 (+ 2665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:51.947752 (+ 4043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:51.947763 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:51.947785 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:51.948482 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:51.948897 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:51.948905 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:51.948911 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:51.949239 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:51.949817 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:51.949828 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:51.950359 (+ 531us) client_negotiation.cc:769] Sending connection context
0811 01:44:51.950701 (+ 342us) client_negotiation.cc:240] Negotiation successful
0811 01:44:51.951390 (+ 689us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":408,"threads_started":1}
I20260811 01:44:51.954810 29273 heartbeater.cc:347] Connected to a master server at 127.21.214.126:34375
I20260811 01:44:51.955178 29273 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:51.956022 29273 heartbeater.cc:511] Master 127.21.214.126:34375 requested a full tablet report, sending...
I20260811 01:44:51.957785 28895 ts_manager.cc:194] Registered new tserver with Master: 1f348f0d3efc4888987bc8dca670f075 (127.21.214.66:42851)
I20260811 01:44:51.958726 28895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:45007
WARNING: no policy specified for kudu/127.21.214.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
I20260811 01:44:51.998090 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:34375
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:39935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:52.358036 29280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:52.358403 29280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:52.358575 29280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:52.366161 29280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:52.366333 29280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:52.366530 29280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:44:52.377000 29280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:39935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:34375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:52.378530 29280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:52.380012 29280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 01:44:52.381963 29120 heartbeater.cc:503] Master 127.21.214.126:34375 was elected leader, sending a full tablet report...
W20260811 01:44:52.396140 29286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:52.396085 29287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:52.396368 29289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:52.396992 29280 server_base.cc:1034] running on GCE node
I20260811 01:44:52.397676 29280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:52.398710 29280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:52.399952 29280 hybrid_clock.cc:648] HybridClock initialized: now 1786412692399895 us; error 48 us; skew 500 ppm
I20260811 01:44:52.400285 29280 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:52 dist-test-slave-9gcw krb5kdc[28848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412692, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:52.404075 29280 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:44:52.405901 29280 webserver.cc:460] Webserver started at http://127.21.214.67:42301/ using document root <none> and password file <none>
I20260811 01:44:52.406666 29280 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:52.406776 29280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:52.407064 29280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:52.410269 29280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "6e1416bbe88c41e78b337356bb65b069"
format_stamp: "Formatted at 2026-08-11 01:44:52 on dist-test-slave-9gcw"
server_key: "b0abef7e2f7b8c4552110b667f5c87d0"
server_key_iv: "2fa6b6a0ba9064acf0ed9debad832f47"
server_key_version: "encryptionkey@0"
I20260811 01:44:52.411096 29280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "6e1416bbe88c41e78b337356bb65b069"
format_stamp: "Formatted at 2026-08-11 01:44:52 on dist-test-slave-9gcw"
server_key: "b0abef7e2f7b8c4552110b667f5c87d0"
server_key_iv: "2fa6b6a0ba9064acf0ed9debad832f47"
server_key_version: "encryptionkey@0"
I20260811 01:44:52.416522 29280 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 01:44:52.420331 29296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:52.421875 29280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 01:44:52.422021 29280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "6e1416bbe88c41e78b337356bb65b069"
format_stamp: "Formatted at 2026-08-11 01:44:52 on dist-test-slave-9gcw"
server_key: "b0abef7e2f7b8c4552110b667f5c87d0"
server_key_iv: "2fa6b6a0ba9064acf0ed9debad832f47"
server_key_version: "encryptionkey@0"
I20260811 01:44:52.422210 29280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:52.444681 29280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:52.449121 29280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:52.449450 29280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:52.450515 29280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:52.451977 29280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:52.452088 29280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:52.452210 29280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:52.452281 29280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:52.487862 29280 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:39563
I20260811 01:44:52.487949 29428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:39563 every 8 connection(s)
I20260811 01:44:52.489331 29280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:44:52.495735 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 29280
I20260811 01:44:52.495962 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:44:52.496455 22361 external_mini_cluster.cc:1428] Setting key 9a81c5540551a66f783b214c5576adfa
Aug 11 01:44:52 dist-test-slave-9gcw krb5kdc[28848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412692, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:44:52.512359 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:52.492429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:32873 (local address 127.21.214.126:34375)
0811 01:44:52.492926 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:52.492931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:52.493906 (+ 975us) server_negotiation.cc:407] Connection header received
0811 01:44:52.495816 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:52.495822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:52.495934 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:52.496075 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:52.497922 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:52.499078 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:52.501002 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:52.501498 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:52.501607 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:52.505491 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:52.505497 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:52.505501 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:52.505537 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:52.508476 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:52.509450 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:52.509455 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:52.509457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:52.509592 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:52.510142 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:52.510146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:52.510149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:52.510528 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:52.510674 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:52.511722 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0811 01:44:52.511976 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":257,"threads_started":1}
I20260811 01:44:52.513499 29432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:52.492841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:34375 (local address 127.21.214.67:32873)
0811 01:44:52.493722 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:52.493768 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:52.495514 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:52.496336 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:52.496345 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:52.496970 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:52.497724 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:52.497743 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:52.499248 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:52.499257 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:52.500856 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:52.500870 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:52.501731 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:52.501738 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:52.501953 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:52.502645 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:52.502671 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:52.505253 (+ 2582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:52.508671 (+ 3418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:52.508680 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:52.508697 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:52.509323 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:52.509768 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:52.509772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:52.509775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:52.509992 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:52.510697 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:52.510706 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:52.511194 (+ 488us) client_negotiation.cc:769] Sending connection context
0811 01:44:52.511566 (+ 372us) client_negotiation.cc:240] Negotiation successful
0811 01:44:52.512111 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":375,"threads_started":1}
I20260811 01:44:52.514976 29429 heartbeater.cc:347] Connected to a master server at 127.21.214.126:34375
I20260811 01:44:52.515265 29429 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:52.515973 29429 heartbeater.cc:511] Master 127.21.214.126:34375 requested a full tablet report, sending...
I20260811 01:44:52.517444 28895 ts_manager.cc:194] Registered new tserver with Master: 6e1416bbe88c41e78b337356bb65b069 (127.21.214.67:39563)
I20260811 01:44:52.518231 28895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:32873
I20260811 01:44:52.525046 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/test-admin.keytab.
Aug 11 01:44:52 dist-test-slave-9gcw krb5kdc[28848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:52.559197 22361 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 01:44:52.564898 29437 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:44:52.562123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34604 (local address 127.21.214.65:38403)
0811 01:44:52.562866 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:52.562874 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:52.562901 (+ 27us) server_negotiation.cc:407] Connection header received
0811 01:44:52.562965 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:52.562972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:52.563194 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:52.563379 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:52.564419 (+ 1040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34604: BlockingRecv error: recv got EOF from 127.0.0.1:34604 (error 108)
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":380,"threads_started":1}
W20260811 01:44:52.565013 22361 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/krb5cc' of type FILE
Aug 11 01:44:52 dist-test-slave-9gcw krb5kdc[28848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:52.566844 22361 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 01:44:52 dist-test-slave-9gcw krb5kdc[28848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:52.568305 22361 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 01:44:52 dist-test-slave-9gcw krb5kdc[28848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:44:52.584753 29437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:52.570179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34618 (local address 127.21.214.65:38403)
0811 01:44:52.570427 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:52.570435 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:52.570660 (+ 225us) server_negotiation.cc:407] Connection header received
0811 01:44:52.570854 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:52.570859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:52.570936 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:52.571029 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:52.572072 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:52.573297 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:52.574671 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:52.575263 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:52.575382 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:52.579167 (+ 3785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:52.579180 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:52.579195 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:52.579228 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:52.582127 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:52.582831 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:52.582838 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:52.582840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:52.582957 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:52.583304 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:52.583307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:52.583309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:52.583662 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:52.583895 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:52.584168 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 01:44:52.584469 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
W20260811 01:44:52.592149 29437 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:44:52.589868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34632 (local address 127.21.214.65:38403)
0811 01:44:52.590165 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:52.590173 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:52.590480 (+ 307us) server_negotiation.cc:407] Connection header received
0811 01:44:52.590767 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:52.590771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:52.590866 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:52.590989 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:52.591874 (+ 885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34632: BlockingRecv error: recv got EOF from 127.0.0.1:34632 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
W20260811 01:44:52.592353 22361 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/krb5cc' of type FILE
Aug 11 01:44:52 dist-test-slave-9gcw krb5kdc[28848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:52.594048 22361 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 01:44:52 dist-test-slave-9gcw krb5kdc[28848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:52.595412 22361 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 01:44:52 dist-test-slave-9gcw krb5kdc[28848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:44:52.609715 29437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:52.596673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34648 (local address 127.21.214.65:38403)
0811 01:44:52.596860 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:52.596869 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:52.597287 (+ 418us) server_negotiation.cc:407] Connection header received
0811 01:44:52.597410 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:52.597414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:52.597468 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:52.597546 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:52.598508 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:52.599228 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:52.600407 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:52.600832 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:52.600929 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:52.604590 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:52.604595 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:52.604598 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:52.604632 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:52.607340 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:52.607943 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:52.607947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:52.607949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:52.608070 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:52.608411 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:52.608415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:52.608417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:52.608676 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:52.608887 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:52.609112 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 01:44:52.609424 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 01:44:52.614248 29437 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:44:52.612454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34654 (local address 127.21.214.65:38403)
0811 01:44:52.612649 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:52.612656 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:52.612834 (+ 178us) server_negotiation.cc:407] Connection header received
0811 01:44:52.613018 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:52.613023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:52.613097 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:52.613232 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:52.613997 (+ 765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34654: BlockingRecv error: recv got EOF from 127.0.0.1:34654 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 01:44:52.615005 22361 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786412636131983-22361-0/krb5kdc/krb5cc)
Aug 11 01:44:52 dist-test-slave-9gcw krb5kdc[28848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:52.616755 22361 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 01:44:52 dist-test-slave-9gcw krb5kdc[28848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:52.618119 22361 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 01:44:52 dist-test-slave-9gcw krb5kdc[28848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:44:52.634060 29437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:52.619621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34662 (local address 127.21.214.65:38403)
0811 01:44:52.619924 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:52.619930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:52.620052 (+ 122us) server_negotiation.cc:407] Connection header received
0811 01:44:52.620266 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:52.620271 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:52.620331 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:52.620422 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:52.621465 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:52.622255 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:52.623478 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:52.623912 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:52.624012 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:52.628355 (+ 4343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:52.628365 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:52.628370 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:52.628415 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:52.631243 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:52.631960 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:52.631964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:52.631967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:52.632102 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:52.632600 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:52.632604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:52.632606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:52.632934 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:52.633295 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:52.633471 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 01:44:52.633693 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 01:44:52.635911 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 28971
I20260811 01:44:52.649626 29437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:52.636694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34674 (local address 127.21.214.65:38403)
0811 01:44:52.636932 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:52.636938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:52.637292 (+ 354us) server_negotiation.cc:407] Connection header received
0811 01:44:52.637455 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:52.637458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:52.637516 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:52.637595 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:52.638943 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:52.639619 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:52.640973 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:52.641513 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:52.641627 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:52.643803 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:52.643808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:52.643811 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:52.643842 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:52.646639 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:52.647461 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:52.647465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:52.647468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:52.647605 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:52.648043 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:52.648046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:52.648048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:52.648379 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:52.648605 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:52.649008 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 01:44:52.649270 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 01:44:52.650238 29115 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:52.742712 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 28971
I20260811 01:44:52.764817 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 29127
Aug 11 01:44:52 dist-test-slave-9gcw krb5kdc[28848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.66@KRBTEST.COM
I20260811 01:44:52.781354 29453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:52.765930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56668 (local address 127.21.214.66:42851)
0811 01:44:52.766480 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:52.766488 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:52.766515 (+ 27us) server_negotiation.cc:407] Connection header received
0811 01:44:52.766576 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:52.766583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:52.766835 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:52.766965 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:52.768180 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:52.769375 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:52.770698 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:52.771273 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:52.771381 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:52.775503 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:52.775514 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:52.775531 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:52.775566 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:52.778637 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:52.779323 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:52.779330 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:52.779332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:52.779455 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:52.779798 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:52.779801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:52.779803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:52.780150 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:52.780384 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:52.780687 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 01:44:52.780916 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":287,"threads_started":1}
I20260811 01:44:52.782819 29268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:52.868427 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 29127
I20260811 01:44:52.891351 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 29280
Aug 11 01:44:52 dist-test-slave-9gcw krb5kdc[28848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.67@KRBTEST.COM
I20260811 01:44:52.908497 29455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:52.892641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33494 (local address 127.21.214.67:39563)
0811 01:44:52.893638 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:52.893648 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:52.893689 (+ 41us) server_negotiation.cc:407] Connection header received
0811 01:44:52.893757 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:52.893764 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:52.893988 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:52.894134 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:52.895232 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:52.896381 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:52.897752 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:52.898397 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:52.898517 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:52.902680 (+ 4163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:52.902692 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:52.902708 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:52.902750 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:52.905959 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:52.906590 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:52.906596 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:52.906599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:52.906725 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:52.907058 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:52.907061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:52.907063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:52.907418 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:52.907653 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:52.907891 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 01:44:52.908148 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":834,"thread_start_us":660,"threads_started":1}
I20260811 01:44:52.910003 29424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:53.002759 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 29280
I20260811 01:44:53.024143 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 28861
I20260811 01:44:53.025441 28931 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:53.132889 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 28861
2026-08-11T01:44:53Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4531 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:44:53 dist-test-slave-9gcw krb5kdc[29460](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:44:53 dist-test-slave-9gcw krb5kdc[29460](info): set up 2 sockets
Aug 11 01:44:53 dist-test-slave-9gcw krb5kdc[29460](info): commencing operation
krb5kdc: starting...
W20260811 01:44:54.356104 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:44:54 dist-test-slave-9gcw krb5kdc[29460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:44:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:44:54Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.126.keytab.
Entry for principal oryx/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.126.keytab.
I20260811 01:44:54.631424 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:41815
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.126.keytab
--principal=oryx/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:45607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:41815
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:54.989037 29473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:54.989498 29473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:54.989666 29473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:54.997414 29473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:44:54.997560 29473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:54.997679 29473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:54.997781 29473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:44:54.997864 29473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:44:55.008450 29473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:45607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:41815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.126.keytab
--principal=oryx/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:41815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:55.010365 29473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:55.011904 29473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:55.026093 29478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:55.025995 29482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:55.026043 29480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:55.027096 29473 server_base.cc:1034] running on GCE node
I20260811 01:44:55.027992 29473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:55.029774 29473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:55.031041 29473 hybrid_clock.cc:648] HybridClock initialized: now 1786412695030980 us; error 52 us; skew 500 ppm
I20260811 01:44:55.031466 29473 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:55 dist-test-slave-9gcw krb5kdc[29460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412695, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:55.035111 29473 init.cc:377] Logged in from keytab as oryx/127.21.214.126@KRBTEST.COM (short username oryx)
I20260811 01:44:55.036880 29473 webserver.cc:460] Webserver started at http://127.21.214.126:41297/ using document root <none> and password file <none>
I20260811 01:44:55.037791 29473 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:55.038105 29473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:55.038703 29473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:55.042419 29473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "662c631857f64f2c96f1a494f2e6c01d"
format_stamp: "Formatted at 2026-08-11 01:44:55 on dist-test-slave-9gcw"
server_key: "172ce38cfcd7f76e313d5c80e2804b9f"
server_key_iv: "6fd635979277c81d1eab45c1f0f54da0"
server_key_version: "encryptionkey@0"
I20260811 01:44:55.043331 29473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "662c631857f64f2c96f1a494f2e6c01d"
format_stamp: "Formatted at 2026-08-11 01:44:55 on dist-test-slave-9gcw"
server_key: "172ce38cfcd7f76e313d5c80e2804b9f"
server_key_iv: "6fd635979277c81d1eab45c1f0f54da0"
server_key_version: "encryptionkey@0"
I20260811 01:44:55.049578 29473 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260811 01:44:55.053634 29492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:55.055117 29473 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 01:44:55.055377 29473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "662c631857f64f2c96f1a494f2e6c01d"
format_stamp: "Formatted at 2026-08-11 01:44:55 on dist-test-slave-9gcw"
server_key: "172ce38cfcd7f76e313d5c80e2804b9f"
server_key_iv: "6fd635979277c81d1eab45c1f0f54da0"
server_key_version: "encryptionkey@0"
I20260811 01:44:55.055612 29473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:55.089232 29473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:55.094074 29473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:55.094411 29473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:55.114300 29473 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:41815
I20260811 01:44:55.114343 29550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:41815 every 8 connection(s)
I20260811 01:44:55.115841 29473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:44:55.119107 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 29473
I20260811 01:44:55.119349 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:44:55.120280 22361 external_mini_cluster.cc:1428] Setting key 3d06c9a6d6fddd441b1776aac8aa61b5
I20260811 01:44:55.120571 29551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:55.136656 29551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d: Bootstrap starting.
Aug 11 01:44:55 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM
I20260811 01:44:55.140887 29551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:55.142400 29551 log.cc:824] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:55.146517 29551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d: No bootstrap required, opened a new log
I20260811 01:44:55.148432 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:55.122683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38772 (local address 127.21.214.126:41815)
0811 01:44:55.123597 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:55.123613 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:55.123663 (+ 50us) server_negotiation.cc:407] Connection header received
0811 01:44:55.125865 (+ 2202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:55.125892 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:55.126342 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:55.126725 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:55.128693 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:55.130194 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:55.132014 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:55.132807 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:55.132971 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:55.138609 (+ 5638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:55.138623 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:55.138645 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:55.138701 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:55.142902 (+ 4201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:55.144136 (+ 1234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:55.144147 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:55.144157 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:55.144409 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:55.144929 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:55.144934 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:55.144937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:55.145740 (+ 803us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:55.146066 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:55.146482 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 01:44:55.146992 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":434,"threads_started":1}
I20260811 01:44:55.155751 29551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } }
I20260811 01:44:55.156132 29551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:55.156208 29551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 662c631857f64f2c96f1a494f2e6c01d, State: Initialized, Role: FOLLOWER
I20260811 01:44:55.156718 29551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } }
I20260811 01:44:55.156929 29551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:44:55.157033 29551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:44:55.157225 29551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:55.159478 29551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } }
I20260811 01:44:55.160151 29551 leader_election.cc:302] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 662c631857f64f2c96f1a494f2e6c01d; no voters:
I20260811 01:44:55.160866 29551 leader_election.cc:288] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:44:55.161005 29558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:44:55.161811 29558 raft_consensus.cc:686] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [term 1 LEADER]: Becoming Leader. State: Replica: 662c631857f64f2c96f1a494f2e6c01d, State: Running, Role: LEADER
I20260811 01:44:55.162355 29558 consensus_queue.cc:237] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } }
I20260811 01:44:55.163169 29551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:44:55.165354 29559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "662c631857f64f2c96f1a494f2e6c01d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } }
I20260811 01:44:55.165814 29559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:55.166255 29561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 662c631857f64f2c96f1a494f2e6c01d. Latest consensus state: current_term: 1 leader_uuid: "662c631857f64f2c96f1a494f2e6c01d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } }
I20260811 01:44:55.166640 29561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:55.167574 29566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:44:55.172408 29566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:44:55.182471 29566 catalog_manager.cc:1347] Generated new cluster ID: 61662ac4f271493a9ff65577e0e6c353
I20260811 01:44:55.182641 29566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:44:55.199924 29566 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:44:55.202105 29566 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:44:55.213974 29566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d: Generated new TSK 0
I20260811 01:44:55.215224 29566 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.65.keytab.
Entry for principal oryx/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.65.keytab.
I20260811 01:44:55.282370 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:41815
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.65.keytab
--principal=oryx/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:45607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:55.638360 29582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:55.638687 29582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:55.638876 29582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:55.646705 29582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:55.646842 29582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:55.647105 29582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:44:55.657786 29582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:45607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.65.keytab
--principal=oryx/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:41815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:55.659289 29582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:55.660856 29582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:55.676200 29588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:55.676249 29589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:55.676326 29594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:55.677232 29582 server_base.cc:1034] running on GCE node
I20260811 01:44:55.677943 29582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:55.679093 29582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:55.680362 29582 hybrid_clock.cc:648] HybridClock initialized: now 1786412695680285 us; error 63 us; skew 500 ppm
I20260811 01:44:55.680764 29582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:55 dist-test-slave-9gcw krb5kdc[29460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412695, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:55.688834 29582 init.cc:377] Logged in from keytab as oryx/127.21.214.65@KRBTEST.COM (short username oryx)
I20260811 01:44:55.690557 29582 webserver.cc:460] Webserver started at http://127.21.214.65:35537/ using document root <none> and password file <none>
I20260811 01:44:55.691277 29582 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:55.691387 29582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:55.691655 29582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:55.694972 29582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "26467dcaea844348b216edcec09ec9eb"
format_stamp: "Formatted at 2026-08-11 01:44:55 on dist-test-slave-9gcw"
server_key: "6ca4337a8478d6a3736b57a5e0ad116c"
server_key_iv: "fe74667a67cbe8a9873136b12116935d"
server_key_version: "encryptionkey@0"
I20260811 01:44:55.695782 29582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "26467dcaea844348b216edcec09ec9eb"
format_stamp: "Formatted at 2026-08-11 01:44:55 on dist-test-slave-9gcw"
server_key: "6ca4337a8478d6a3736b57a5e0ad116c"
server_key_iv: "fe74667a67cbe8a9873136b12116935d"
server_key_version: "encryptionkey@0"
I20260811 01:44:55.701584 29582 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 01:44:55.705335 29601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:55.706801 29582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:44:55.706944 29582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "26467dcaea844348b216edcec09ec9eb"
format_stamp: "Formatted at 2026-08-11 01:44:55 on dist-test-slave-9gcw"
server_key: "6ca4337a8478d6a3736b57a5e0ad116c"
server_key_iv: "fe74667a67cbe8a9873136b12116935d"
server_key_version: "encryptionkey@0"
I20260811 01:44:55.707120 29582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:55.731796 29582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:55.736655 29582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:55.736922 29582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:55.737926 29582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:55.739344 29582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:55.739429 29582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:55.739517 29582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:55.739562 29582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:55.774925 29582 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:34917
I20260811 01:44:55.774979 29722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:34917 every 8 connection(s)
I20260811 01:44:55.776402 29582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:44:55.779999 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 29582
I20260811 01:44:55.780230 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:44:55.780925 22361 external_mini_cluster.cc:1428] Setting key 468e1950ae52fc8959417d8fca873b46
Aug 11 01:44:55 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412695, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM
I20260811 01:44:55.800001 29725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:55.779163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:57701 (local address 127.21.214.126:41815)
0811 01:44:55.779692 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:55.779700 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:55.781026 (+ 1326us) server_negotiation.cc:407] Connection header received
0811 01:44:55.783186 (+ 2160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:55.783192 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:55.783305 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:55.783427 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:55.785025 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:55.786195 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:55.788025 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:55.788752 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:55.788914 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:55.793342 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:55.793347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:55.793351 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:55.793388 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:55.796292 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:55.797405 (+ 1113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:55.797409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:55.797411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:55.797535 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:55.798125 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:55.798128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:55.798130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:55.798398 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:55.798512 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:55.799539 (+ 1027us) server_negotiation.cc:299] Negotiation successful
0811 01:44:55.799727 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":265,"threads_started":1}
I20260811 01:44:55.801951 29726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:55.779810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.65:57701)
0811 01:44:55.780760 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:55.780807 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:55.782876 (+ 2069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:55.783640 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:55.783648 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:55.784190 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:55.784830 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:55.784853 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:55.786415 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:55.786419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:55.787852 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:55.787867 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:55.788992 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:55.788997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:55.789311 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:55.790141 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:55.790190 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:55.793028 (+ 2838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:55.796461 (+ 3433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:55.796475 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:55.796498 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:55.797282 (+ 784us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:55.797658 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:55.797664 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:55.797669 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:55.798003 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:55.798539 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:55.798551 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:55.799078 (+ 527us) client_negotiation.cc:769] Sending connection context
0811 01:44:55.799441 (+ 363us) client_negotiation.cc:240] Negotiation successful
0811 01:44:55.800158 (+ 717us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":415,"threads_started":1}
I20260811 01:44:55.803808 29723 heartbeater.cc:347] Connected to a master server at 127.21.214.126:41815
I20260811 01:44:55.804149 29723 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:55.805011 29723 heartbeater.cc:511] Master 127.21.214.126:41815 requested a full tablet report, sending...
I20260811 01:44:55.807366 29512 ts_manager.cc:194] Registered new tserver with Master: 26467dcaea844348b216edcec09ec9eb (127.21.214.65:34917)
WARNING: no policy specified for oryx/127.21.214.66@KRBTEST.COM; defaulting to no policy
I20260811 01:44:55.808899 29512 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:57701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.214.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.66.keytab.
Entry for principal oryx/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.66.keytab.
I20260811 01:44:55.844033 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:41815
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.66.keytab
--principal=oryx/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:45607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:56.204814 29729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:56.205214 29729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:56.205382 29729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:56.213445 29729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:56.213625 29729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:56.213807 29729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:44:56.224304 29729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:45607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.66.keytab
--principal=oryx/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:41815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:56.225888 29729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:56.227340 29729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:56.241490 29736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:56.241453 29735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:56.241524 29738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:56.242460 29729 server_base.cc:1034] running on GCE node
I20260811 01:44:56.243117 29729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:56.244191 29729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:56.245440 29729 hybrid_clock.cc:648] HybridClock initialized: now 1786412696245396 us; error 73 us; skew 500 ppm
I20260811 01:44:56.245781 29729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:56 dist-test-slave-9gcw krb5kdc[29460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412696, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:56.249220 29729 init.cc:377] Logged in from keytab as oryx/127.21.214.66@KRBTEST.COM (short username oryx)
I20260811 01:44:56.250922 29729 webserver.cc:460] Webserver started at http://127.21.214.66:37873/ using document root <none> and password file <none>
I20260811 01:44:56.251662 29729 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:56.251818 29729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:56.252141 29729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:56.255215 29729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "56802ed9c3784965b20f771a4840de0b"
format_stamp: "Formatted at 2026-08-11 01:44:56 on dist-test-slave-9gcw"
server_key: "a61a6f6c23ef3c1bf511648842a4e034"
server_key_iv: "b06031b05f3e76ddfce448e29126fc77"
server_key_version: "encryptionkey@0"
I20260811 01:44:56.256307 29729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "56802ed9c3784965b20f771a4840de0b"
format_stamp: "Formatted at 2026-08-11 01:44:56 on dist-test-slave-9gcw"
server_key: "a61a6f6c23ef3c1bf511648842a4e034"
server_key_iv: "b06031b05f3e76ddfce448e29126fc77"
server_key_version: "encryptionkey@0"
I20260811 01:44:56.262120 29729 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 01:44:56.266115 29745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:56.267664 29729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:44:56.267870 29729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "56802ed9c3784965b20f771a4840de0b"
format_stamp: "Formatted at 2026-08-11 01:44:56 on dist-test-slave-9gcw"
server_key: "a61a6f6c23ef3c1bf511648842a4e034"
server_key_iv: "b06031b05f3e76ddfce448e29126fc77"
server_key_version: "encryptionkey@0"
I20260811 01:44:56.268093 29729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:56.305702 29729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:56.310410 29729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:56.310735 29729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:56.311911 29729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:56.313416 29729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:56.313493 29729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:56.313584 29729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:56.313628 29729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:56.348945 29729 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:38921
I20260811 01:44:56.349066 29876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:38921 every 8 connection(s)
I20260811 01:44:56.350310 29729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:44:56.352017 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 29729
I20260811 01:44:56.352214 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:44:56.352828 22361 external_mini_cluster.cc:1428] Setting key 8c30454609c51631df3b4ea2688eca1e
Aug 11 01:44:56 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412696, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM
I20260811 01:44:56.377269 29880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:56.353455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:38865 (local address 127.21.214.126:41815)
0811 01:44:56.353951 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:56.353958 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:56.355614 (+ 1656us) server_negotiation.cc:407] Connection header received
0811 01:44:56.358643 (+ 3029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:56.358648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:56.358721 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:56.358824 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:56.361129 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:56.361923 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:56.364510 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:56.364927 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:56.365042 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:56.369196 (+ 4154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:56.369206 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:56.369213 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:56.369268 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:56.373231 (+ 3963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:56.374159 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:56.374168 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:56.374174 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:56.374359 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:56.374869 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:56.374877 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:56.374883 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:56.375367 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:56.375532 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:56.376405 (+ 873us) server_negotiation.cc:299] Negotiation successful
0811 01:44:56.376694 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":249,"threads_started":1}
I20260811 01:44:56.378139 29881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:56.354168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.66:38865)
0811 01:44:56.355451 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:56.355501 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:56.358304 (+ 2803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:56.359127 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:56.359141 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:56.360021 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:56.360930 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:56.360962 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:56.362089 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:56.362097 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:56.364358 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:56.364377 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:56.365202 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:56.365212 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:56.365560 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:56.366409 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:56.366437 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:56.368894 (+ 2457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:56.373431 (+ 4537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:56.373439 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:56.373455 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:56.374009 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:56.374490 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:56.374494 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:56.374496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:56.374735 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:56.375513 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:56.375520 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:56.375975 (+ 455us) client_negotiation.cc:769] Sending connection context
0811 01:44:56.376242 (+ 267us) client_negotiation.cc:240] Negotiation successful
0811 01:44:56.376708 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"thread_start_us":692,"threads_started":1}
I20260811 01:44:56.379823 29878 heartbeater.cc:347] Connected to a master server at 127.21.214.126:41815
I20260811 01:44:56.380178 29878 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:56.380986 29878 heartbeater.cc:511] Master 127.21.214.126:41815 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.21.214.67@KRBTEST.COM; defaulting to no policy
I20260811 01:44:56.382779 29512 ts_manager.cc:194] Registered new tserver with Master: 56802ed9c3784965b20f771a4840de0b (127.21.214.66:38921)
I20260811 01:44:56.383711 29512 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:38865
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.214.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.67.keytab.
Entry for principal oryx/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.67.keytab.
I20260811 01:44:56.416417 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:41815
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.67.keytab
--principal=oryx/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:45607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:56.772451 29884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:56.772763 29884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:56.772941 29884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:56.781479 29884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:56.781603 29884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:56.781828 29884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:44:56.791973 29884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:45607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.67.keytab
--principal=oryx/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:41815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:56.793473 29884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:56.794912 29884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:56.809891 29894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:56.809933 29890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:56.809985 29889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:56.812402 29723 heartbeater.cc:503] Master 127.21.214.126:41815 was elected leader, sending a full tablet report...
I20260811 01:44:56.816308 29884 server_base.cc:1034] running on GCE node
I20260811 01:44:56.817022 29884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:56.818126 29884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:56.819353 29884 hybrid_clock.cc:648] HybridClock initialized: now 1786412696819302 us; error 42 us; skew 500 ppm
I20260811 01:44:56.819715 29884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:56 dist-test-slave-9gcw krb5kdc[29460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412696, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:56.823262 29884 init.cc:377] Logged in from keytab as oryx/127.21.214.67@KRBTEST.COM (short username oryx)
I20260811 01:44:56.824807 29884 webserver.cc:460] Webserver started at http://127.21.214.67:44563/ using document root <none> and password file <none>
I20260811 01:44:56.825652 29884 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:56.825815 29884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:56.826156 29884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:56.829425 29884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "39a96547f8ec4bf08372b8ce43c159a7"
format_stamp: "Formatted at 2026-08-11 01:44:56 on dist-test-slave-9gcw"
server_key: "3e8c87ee95538c52979a42ef24da4bc8"
server_key_iv: "7a217b4762740511b5e802028d8596dd"
server_key_version: "encryptionkey@0"
I20260811 01:44:56.830309 29884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "39a96547f8ec4bf08372b8ce43c159a7"
format_stamp: "Formatted at 2026-08-11 01:44:56 on dist-test-slave-9gcw"
server_key: "3e8c87ee95538c52979a42ef24da4bc8"
server_key_iv: "7a217b4762740511b5e802028d8596dd"
server_key_version: "encryptionkey@0"
I20260811 01:44:56.836066 29884 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 01:44:56.839883 29903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:56.841428 29884 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:44:56.841624 29884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "39a96547f8ec4bf08372b8ce43c159a7"
format_stamp: "Formatted at 2026-08-11 01:44:56 on dist-test-slave-9gcw"
server_key: "3e8c87ee95538c52979a42ef24da4bc8"
server_key_iv: "7a217b4762740511b5e802028d8596dd"
server_key_version: "encryptionkey@0"
I20260811 01:44:56.841856 29884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:56.862656 29884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:56.867700 29884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:56.868096 29884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:56.869352 29884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:56.870903 29884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:56.871013 29884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:56.871171 29884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:56.871255 29884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:56.911906 29884 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:38341
I20260811 01:44:56.911947 30031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:38341 every 8 connection(s)
I20260811 01:44:56.913503 29884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:44:56.914515 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 29884
I20260811 01:44:56.914723 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:44:56.915417 22361 external_mini_cluster.cc:1428] Setting key 14a6adc4bf79a678bdb068c50ef061e2
Aug 11 01:44:56 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412696, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM
I20260811 01:44:56.939044 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:56.916698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:56587 (local address 127.21.214.126:41815)
0811 01:44:56.917154 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:56.917190 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:56.918567 (+ 1377us) server_negotiation.cc:407] Connection header received
0811 01:44:56.920659 (+ 2092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:56.920663 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:56.920737 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:56.920843 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:56.922463 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:56.923973 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:56.926805 (+ 2832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:56.927600 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:56.927743 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:56.932915 (+ 5172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:56.932922 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:56.932928 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:56.932976 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:56.935931 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:56.936681 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:56.936687 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:56.936691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:56.936852 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:56.937324 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:56.937327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:56.937330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:56.937626 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:56.937789 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:56.938521 (+ 732us) server_negotiation.cc:299] Negotiation successful
0811 01:44:56.938741 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":198,"threads_started":1}
I20260811 01:44:56.940032 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:56.917414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.67:56587)
0811 01:44:56.918374 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:56.918439 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:56.920347 (+ 1908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:56.921062 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:56.921070 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:56.921713 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:56.922295 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:56.922313 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:56.924163 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:56.924171 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:56.926612 (+ 2441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:56.926632 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:56.927908 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:56.927922 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:56.928304 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:56.929223 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:56.929292 (+ 69us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:56.932657 (+ 3365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:56.936074 (+ 3417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:56.936081 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:56.936097 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:56.936543 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:56.936957 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:56.936960 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:56.936962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:56.937201 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:56.937764 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:56.937798 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:56.938142 (+ 344us) client_negotiation.cc:769] Sending connection context
0811 01:44:56.938387 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 01:44:56.938815 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":382,"threads_started":1}
I20260811 01:44:56.941821 30032 heartbeater.cc:347] Connected to a master server at 127.21.214.126:41815
I20260811 01:44:56.942148 30032 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:56.942981 30032 heartbeater.cc:511] Master 127.21.214.126:41815 requested a full tablet report, sending...
I20260811 01:44:56.944679 29510 ts_manager.cc:194] Registered new tserver with Master: 39a96547f8ec4bf08372b8ce43c159a7 (127.21.214.67:38341)
I20260811 01:44:56.945966 29510 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:56587
I20260811 01:44:56.946056 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.214.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.214.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.214.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.125.keytab.
Entry for principal oryx/127.21.214.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.125.keytab.
I20260811 01:44:57.010459 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.125:41479
--webserver_interface=127.21.214.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.125.keytab
--principal=oryx/127.21.214.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:45607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:41815,127.21.214.125:41479
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:57.369701 30038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:57.370005 30038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:57.370206 30038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:57.377892 30038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:44:57.378011 30038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:57.378100 30038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:57.378214 30038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:44:57.378276 30038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:44:57.387277 29878 heartbeater.cc:503] Master 127.21.214.126:41815 was elected leader, sending a full tablet report...
I20260811 01:44:57.388955 30038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:45607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:41815,127.21.214.125:41479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.125.keytab
--principal=oryx/127.21.214.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.125:41479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.214.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
W20260811 01:44:57.389479 30038 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.21.214.126:41815,127.21.214.125:41479'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 01:44:57.390591 30038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:57.391911 30038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:57.404428 30044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:57.404537 30049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:57.404615 30045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:57.405488 30038 server_base.cc:1034] running on GCE node
I20260811 01:44:57.406258 30038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:57.407720 30038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:57.408954 30038 hybrid_clock.cc:648] HybridClock initialized: now 1786412697408885 us; error 38 us; skew 500 ppm
I20260811 01:44:57.409368 30038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:57 dist-test-slave-9gcw krb5kdc[29460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412697, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:57.412998 30038 init.cc:377] Logged in from keytab as oryx/127.21.214.125@KRBTEST.COM (short username oryx)
I20260811 01:44:57.414723 30038 webserver.cc:460] Webserver started at http://127.21.214.125:44827/ using document root <none> and password file <none>
I20260811 01:44:57.415479 30038 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:57.415571 30038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:57.415850 30038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:57.419122 30038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/data/instance:
uuid: "bbd0420e2f7342008f02ce52a2778fbf"
format_stamp: "Formatted at 2026-08-11 01:44:57 on dist-test-slave-9gcw"
server_key: "aa26e9cf3750e94c21e8f42687c585fc"
server_key_iv: "6a1509b1f9ab6246e578a7beafccdbc0"
server_key_version: "encryptionkey@0"
I20260811 01:44:57.419948 30038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/wal/instance:
uuid: "bbd0420e2f7342008f02ce52a2778fbf"
format_stamp: "Formatted at 2026-08-11 01:44:57 on dist-test-slave-9gcw"
server_key: "aa26e9cf3750e94c21e8f42687c585fc"
server_key_iv: "6a1509b1f9ab6246e578a7beafccdbc0"
server_key_version: "encryptionkey@0"
I20260811 01:44:57.425638 30038 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 01:44:57.429466 30056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:57.430992 30038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:44:57.431154 30038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/wal
uuid: "bbd0420e2f7342008f02ce52a2778fbf"
format_stamp: "Formatted at 2026-08-11 01:44:57 on dist-test-slave-9gcw"
server_key: "aa26e9cf3750e94c21e8f42687c585fc"
server_key_iv: "6a1509b1f9ab6246e578a7beafccdbc0"
server_key_version: "encryptionkey@0"
I20260811 01:44:57.431350 30038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:57.448156 30038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:57.452745 30038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:57.452989 30038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:57.473528 30038 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.125:41479
I20260811 01:44:57.473562 30114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.125:41479 every 8 connection(s)
I20260811 01:44:57.475088 30038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/data/info.pb
I20260811 01:44:57.478044 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 30038
I20260811 01:44:57.479560 30115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:57.488790 30115 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:44:57 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.214.125@KRBTEST.COM
Aug 11 01:44:57 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412697, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.125@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM
I20260811 01:44:57.503587 30119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:57.480310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45632 (local address 127.21.214.125:41479)
0811 01:44:57.481232 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:57.481250 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:57.481698 (+ 448us) server_negotiation.cc:407] Connection header received
0811 01:44:57.483456 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:57.483479 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:57.483934 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:57.484383 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:57.486065 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.487425 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.488702 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.489529 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.489731 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:57.494885 (+ 5154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:57.494893 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:57.494904 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:57.494941 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:57.499071 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:57.499789 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:57.499796 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:57.499803 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:57.500049 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:57.500495 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:57.500500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:57.500502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:57.501254 (+ 752us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:57.501582 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:57.502058 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 01:44:57.502535 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"spinlock_wait_cycles":17152,"thread_start_us":397,"threads_started":1}
I20260811 01:44:57.508949 30121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:57.491752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38780 (local address 127.21.214.126:41815)
0811 01:44:57.493289 (+ 1537us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:57.493293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:57.493311 (+ 18us) server_negotiation.cc:407] Connection header received
0811 01:44:57.493366 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:57.493369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:57.493441 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:57.493569 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:57.494840 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.495929 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.498189 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.498731 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.498853 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:57.502590 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:57.502595 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:57.502598 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:57.502709 (+ 111us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:57.506202 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:57.507107 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:57.507112 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:57.507114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:57.507244 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:57.507606 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:57.507609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:57.507611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:57.507970 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:57.508088 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:57.508505 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 01:44:57.508687 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1357,"thread_start_us":1221,"threads_started":1}
I20260811 01:44:57.509006 30120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:57.491685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:38780)
0811 01:44:57.492656 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:57.492726 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:57.493000 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:57.493633 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:57.493641 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:57.494370 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:57.494634 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:57.494644 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.496091 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.496096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:57.497999 (+ 1903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:57.498015 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.498944 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.498952 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:57.499155 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:57.499812 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:57.499829 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:57.502371 (+ 2542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:57.506342 (+ 3971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:57.506350 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:57.506356 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:57.506989 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:57.507355 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:57.507358 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:57.507361 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:57.507497 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:57.508095 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:57.508102 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:57.508257 (+ 155us) client_negotiation.cc:769] Sending connection context
0811 01:44:57.508413 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 01:44:57.508715 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":407,"threads_started":1}
I20260811 01:44:57.511049 30115 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:44:57 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412697, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.125@KRBTEST.COM for oryx/127.21.214.125@KRBTEST.COM
I20260811 01:44:57.524102 30120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:57.511833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:41479 (local address 127.0.0.1:45634)
0811 01:44:57.512059 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:57.512077 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:57.512174 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:57.512448 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:57.512451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:57.512863 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:57.513011 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:57.513026 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.514067 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.514071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:57.514977 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:57.514988 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.515705 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.515709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:57.515881 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:57.516544 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:57.516561 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:57.518732 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:57.521837 (+ 3105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:57.521841 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:57.521844 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:57.522215 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:57.522676 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:57.522679 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:57.522682 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:57.522821 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:57.523371 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:57.523374 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:57.523505 (+ 131us) client_negotiation.cc:769] Sending connection context
0811 01:44:57.523615 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 01:44:57.523850 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 01:44:57.524358 30119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:57.512022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45634 (local address 127.21.214.125:41479)
0811 01:44:57.512181 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:57.512186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:57.512202 (+ 16us) server_negotiation.cc:407] Connection header received
0811 01:44:57.512244 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:57.512247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:57.512302 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:57.512379 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:57.513217 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.513914 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.515128 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.515568 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.515686 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:57.518911 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:57.518916 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:57.518919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:57.518947 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:57.521699 (+ 2752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:57.522393 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:57.522398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:57.522400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:57.522568 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:57.522949 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:57.522953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:57.522955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:57.523262 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:57.523383 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:57.523779 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 01:44:57.523971 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 01:44:57.529450 30115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: Bootstrap starting.
I20260811 01:44:57.532627 30115 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:57.533788 30115 log.cc:824] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:57.536813 30115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: No bootstrap required, opened a new log
I20260811 01:44:57.543982 30115 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } }
I20260811 01:44:57.544345 30115 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:57.544445 30115 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbd0420e2f7342008f02ce52a2778fbf, State: Initialized, Role: FOLLOWER
I20260811 01:44:57.545207 30115 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } }
I20260811 01:44:57.546118 30123 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } } }
I20260811 01:44:57.546509 30123 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:44:57.547232 30115 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [sys.catalog]: configured and running, proceeding with master startup.
W20260811 01:44:57.557175 30135 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 01:44:57.557404 30135 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 01:44:57.560304 30038 master_runner.cc:428] Detected that this master bbd0420e2f7342008f02ce52a2778fbf is joining an existing cluster
I20260811 01:44:57.560438 30038 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.214.125:41479
I20260811 01:44:57.589823 30119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:57.572305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45640 (local address 127.21.214.125:41479)
0811 01:44:57.572533 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:57.572542 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:57.573360 (+ 818us) server_negotiation.cc:407] Connection header received
0811 01:44:57.573645 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:57.573652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:57.573759 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:57.573872 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:57.575549 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.576288 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.578412 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.579127 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.579231 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:57.582069 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:57.582077 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:57.582085 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:57.582120 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:57.586222 (+ 4102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:57.587100 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:57.587105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:57.587109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:57.587295 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:57.587737 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:57.587743 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:57.587747 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:57.588106 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:57.588295 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:57.588831 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 01:44:57.589070 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 01:44:57.590991 30142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:57.572261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:41479 (local address 127.0.0.1:45640)
0811 01:44:57.573208 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:57.573232 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:57.573510 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:57.574067 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:57.574073 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:57.574865 (+ 792us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:57.575378 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:57.575394 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.576467 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.576473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:57.578220 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:57.578237 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.579578 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.579584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:57.579773 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:57.580556 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:57.580581 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:57.581868 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:57.586393 (+ 4525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:57.586398 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:57.586403 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:57.586972 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:57.587453 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:57.587457 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:57.587460 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:57.587627 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:57.588385 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:57.588393 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:57.588561 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 01:44:57.589420 (+ 859us) client_negotiation.cc:240] Negotiation successful
0811 01:44:57.589796 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":445,"threads_started":1}
WARNING: no policy specified for oryx/127.21.214.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.214.124@KRBTEST.COM" created.
I20260811 01:44:57.599378 30121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:57.573448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38782 (local address 127.21.214.126:41815)
0811 01:44:57.573637 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:57.573643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:57.580974 (+ 7331us) server_negotiation.cc:407] Connection header received
0811 01:44:57.581505 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:57.581513 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:57.581626 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:57.581706 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:57.583759 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.584993 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.587414 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.588215 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.588333 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:57.592254 (+ 3921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:57.592263 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:57.592277 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:57.592323 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:57.596055 (+ 3732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:57.596816 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:57.596821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:57.596825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:57.596985 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:57.597495 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:57.597500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:57.597504 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:57.597870 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:57.598044 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:57.598579 (+ 535us) server_negotiation.cc:299] Negotiation successful
0811 01:44:57.599007 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 01:44:57.599843 30144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:57.574019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:38782)
0811 01:44:57.580827 (+ 6808us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:57.580850 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:57.581327 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:57.581946 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:57.581953 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:57.583101 (+ 1148us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:57.583552 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:57.583571 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.585275 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.585282 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:57.587234 (+ 1952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:57.587255 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.588666 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.588675 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:57.588886 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:57.590354 (+ 1468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:57.590386 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:57.592033 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:57.596227 (+ 4194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:57.596231 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:57.596233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:57.596666 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:57.597221 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:57.597226 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:57.597228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:57.597366 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:57.598015 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:57.598019 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:57.598268 (+ 249us) client_negotiation.cc:769] Sending connection context
0811 01:44:57.598428 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 01:44:57.599027 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6622,"thread_start_us":6978,"threads_started":1}
Aug 11 01:44:57 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412695, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.126@KRBTEST.COM for oryx/127.21.214.125@KRBTEST.COM
I20260811 01:44:57.629739 30119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:57.608041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45650 (local address 127.21.214.125:41479)
0811 01:44:57.608235 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:57.608242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:57.609830 (+ 1588us) server_negotiation.cc:407] Connection header received
0811 01:44:57.610293 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:57.610299 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:57.610399 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:57.610512 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:57.611805 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.613033 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.615208 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.615941 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.616079 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:57.621082 (+ 5003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:57.621093 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:57.621102 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:57.621192 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:57.625783 (+ 4591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:57.627009 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:57.627018 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:57.627035 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:57.627219 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:57.627809 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:57.627815 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:57.627818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:57.628245 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:57.628415 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:57.629048 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 01:44:57.629325 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 01:44:57.629837 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:57.607882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:41479 (local address 127.0.0.1:45650)
0811 01:44:57.609683 (+ 1801us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:57.609729 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:57.609990 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:57.610586 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:57.610597 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:57.611317 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:57.611542 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:57.611556 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.613233 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.613241 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:57.615040 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:57.615059 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.616143 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.616170 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:57.616421 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:57.617325 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:57.617358 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:57.620797 (+ 3439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:57.626001 (+ 5204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:57.626024 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:57.626038 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:57.626814 (+ 776us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:57.627387 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:57.627396 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:57.627402 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:57.627649 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:57.628418 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:57.628425 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:57.628602 (+ 177us) client_negotiation.cc:769] Sending connection context
0811 01:44:57.628877 (+ 275us) client_negotiation.cc:240] Negotiation successful
0811 01:44:57.629330 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1517,"thread_start_us":1572,"threads_started":1}
I20260811 01:44:57.632356 29510 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.214.125:41479: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf"
member_type: NON_VOTER
last_known_addr {
host: "127.21.214.125"
port: 41479
}
attrs {
promote: true
}
}
dest_uuid: "662c631857f64f2c96f1a494f2e6c01d"
cas_config_opid_index: -1
I20260811 01:44:57.633860 29510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: NON_VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.214.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.124.keytab.
Entry for principal oryx/127.21.214.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.124.keytab.
I20260811 01:44:57.638675 30085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:57.637633 30152 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bbd0420e2f7342008f02ce52a2778fbf (127.21.214.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: NON_VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: true } } }
I20260811 01:44:57.640021 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.124:39589
--webserver_interface=127.21.214.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.124.keytab
--principal=oryx/127.21.214.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:45607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:41815,127.21.214.125:41479,127.21.214.124:39589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:44:57.643086 30152 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.214.125:41479
I20260811 01:44:57.643198 30085 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 1 FOLLOWER]: Refusing update from remote peer 662c631857f64f2c96f1a494f2e6c01d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 01:44:57.644205 30038 master.cc:537] Master@127.21.214.125:41479 shutting down...
I20260811 01:44:57.647495 30152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "662c631857f64f2c96f1a494f2e6c01d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: NON_VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: true } } }
I20260811 01:44:57.647771 30152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:57.658601 30038 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:44:57.659229 30038 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:57.659483 30038 tablet_replica.cc:330] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: stopping tablet replica
I20260811 01:44:57.659916 30151 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: NON_VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:57.671461 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:57.670917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:41479 (local address 127.0.0.1:45654)
0811 01:44:57.671088 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:57.671248 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:41479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 01:44:57.671998 29493 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.214.125:41479: connect: Connection refused (error 111)
I20260811 01:44:57.672621 30038 master.cc:552] Master@127.21.214.125:41479 shutdown complete.
I20260811 01:44:57.672830 30038 master_runner.cc:305] Clearing existing system tablet
I20260811 01:44:57.673406 30038 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:57.673563 30038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:57.682966 30038 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.004s sys 0.003s
I20260811 01:44:57.683676 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:57.683030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:41479 (local address 127.0.0.1:45670)
0811 01:44:57.683346 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:57.683454 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:41479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
W20260811 01:44:57.683933 29493 consensus_peers.cc:487] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d -> Peer bbd0420e2f7342008f02ce52a2778fbf (127.21.214.125:41479): Couldn't send request to peer bbd0420e2f7342008f02ce52a2778fbf. Status: Network error: Client connection negotiation failed: client connection to 127.21.214.125:41479: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 01:44:57.685972 30159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:57.686329 30038 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:57.686611 30038 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 01:44:57.686714 30038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/wal
uuid: "bbd0420e2f7342008f02ce52a2778fbf"
format_stamp: "Formatted at 2026-08-11 01:44:57 on dist-test-slave-9gcw"
server_key: "aa26e9cf3750e94c21e8f42687c585fc"
server_key_iv: "6a1509b1f9ab6246e578a7beafccdbc0"
server_key_version: "encryptionkey@0"
I20260811 01:44:57.692534 30038 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:57.699957 30038 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 01:44:57.700168 30038 log.cc:1196] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 01:44:57.700688 30038 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: Deleting consensus metadata
I20260811 01:44:57.700991 30038 master_runner.cc:315] Copying system tablet from 127.21.214.126:41815
I20260811 01:44:57.704172 30038 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: tablet copy: Beginning tablet copy session from remote peer at address 127.21.214.126:41815
I20260811 01:44:57.721205 30165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:57.705340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:38794)
0811 01:44:57.705878 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:57.705899 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:57.706047 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:57.706664 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:57.706669 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:57.707229 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:57.707517 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:57.707533 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.709021 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.709027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:57.710600 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:57.710616 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.711636 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.711643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:57.711855 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:57.712618 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:57.712643 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:57.713813 (+ 1170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:57.718074 (+ 4261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:57.718080 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:57.718084 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:57.718610 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:57.719084 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:57.719089 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:57.719093 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:57.719267 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:57.719965 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:57.719970 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:57.720156 (+ 186us) client_negotiation.cc:769] Sending connection context
0811 01:44:57.720307 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 01:44:57.720682 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":653,"threads_started":1}
I20260811 01:44:57.721470 30121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:57.705771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38794 (local address 127.21.214.126:41815)
0811 01:44:57.705961 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:57.705967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:57.705992 (+ 25us) server_negotiation.cc:407] Connection header received
0811 01:44:57.706234 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:57.706241 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:57.706347 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:57.706427 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:57.707724 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.708865 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.710771 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:57.711453 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:57.711603 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:57.714046 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:57.714052 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:57.714057 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:57.714098 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:57.717896 (+ 3798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:57.718774 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:57.718779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:57.718783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:57.718957 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:57.719417 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:57.719422 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:57.719425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:57.719822 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:57.719992 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:57.720474 (+ 482us) server_negotiation.cc:299] Negotiation successful
0811 01:44:57.720785 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 01:44:57.722592 29536 tablet_copy_service.cc:140] P 662c631857f64f2c96f1a494f2e6c01d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bbd0420e2f7342008f02ce52a2778fbf ({username='oryx', principal='oryx/127.21.214.125@KRBTEST.COM'} at 127.0.0.1:38794)
I20260811 01:44:57.723105 29536 tablet_copy_service.cc:161] P 662c631857f64f2c96f1a494f2e6c01d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bbd0420e2f7342008f02ce52a2778fbf at {username='oryx', principal='oryx/127.21.214.125@KRBTEST.COM'} at 127.0.0.1:38794: session id = bbd0420e2f7342008f02ce52a2778fbf-00000000000000000000000000000000
I20260811 01:44:57.728350 29536 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 01:44:57.731655 30038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 01:44:57.739171 30038 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: tablet copy: Starting download of 0 data blocks...
I20260811 01:44:57.739607 30038 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: tablet copy: Starting download of 1 WAL segments...
I20260811 01:44:57.743417 30038 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 01:44:57.748359 29536 tablet_copy_service.cc:342] P 662c631857f64f2c96f1a494f2e6c01d: Request end of tablet copy session bbd0420e2f7342008f02ce52a2778fbf-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.214.125@KRBTEST.COM'} at 127.0.0.1:38794
I20260811 01:44:57.748682 29536 tablet_copy_service.cc:433] P 662c631857f64f2c96f1a494f2e6c01d: ending tablet copy session bbd0420e2f7342008f02ce52a2778fbf-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bbd0420e2f7342008f02ce52a2778fbf
I20260811 01:44:57.762009 30038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:57.765208 30166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:57.779726 30168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:57.787225 30038 server_base.cc:1034] running on GCE node
W20260811 01:44:57.788518 30171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:57.789124 30038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:57.789963 30038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:57.791281 30038 hybrid_clock.cc:648] HybridClock initialized: now 1786412697791208 us; error 43 us; skew 500 ppm
I20260811 01:44:57.791555 30038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:57 dist-test-slave-9gcw krb5kdc[29460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412697, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:57.794106 30038 init.cc:377] Logged in from keytab as oryx/127.21.214.125@KRBTEST.COM (short username oryx)
I20260811 01:44:57.795526 30038 webserver.cc:460] Webserver started at http://127.21.214.125:32927/ using document root <none> and password file <none>
I20260811 01:44:57.795948 30038 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:57.796046 30038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:57.799424 30038 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 01:44:57.801961 30178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:57.802456 30038 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 01:44:57.802613 30038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/wal
uuid: "bbd0420e2f7342008f02ce52a2778fbf"
format_stamp: "Formatted at 2026-08-11 01:44:57 on dist-test-slave-9gcw"
server_key: "aa26e9cf3750e94c21e8f42687c585fc"
server_key_iv: "6a1509b1f9ab6246e578a7beafccdbc0"
server_key_version: "encryptionkey@0"
I20260811 01:44:57.802807 30038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:57.820973 30038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:57.825532 30038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:57.854140 30038 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.125:41479
I20260811 01:44:57.854174 30239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.125:41479 every 8 connection(s)
I20260811 01:44:57.855584 30038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-1/data/info.pb
I20260811 01:44:57.861349 30240 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:44:57.865732 30240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: Bootstrap starting.
I20260811 01:44:57.887329 30240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:44:57.887957 30240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: Bootstrap complete.
I20260811 01:44:57.889276 30240 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: NON_VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: true } }
I20260811 01:44:57.889582 30240 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bbd0420e2f7342008f02ce52a2778fbf, State: Initialized, Role: LEARNER
I20260811 01:44:57.889931 30240 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: NON_VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: true } }
I20260811 01:44:57.891286 30240 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:44:57.891013 30246 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: NON_VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: true } } }
I20260811 01:44:57.891660 30246 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [sys.catalog]: This master's current role is: LEARNER
I20260811 01:44:57.905400 30258 catalog_manager.cc:1259] Loaded cluster ID: 61662ac4f271493a9ff65577e0e6c353
I20260811 01:44:57.905535 30258 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: loading cluster ID for follower catalog manager: success
I20260811 01:44:57.909358 30258 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: acquiring CA information for follower catalog manager: success
I20260811 01:44:57.912195 30258 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:44:57.948818 30032 heartbeater.cc:503] Master 127.21.214.126:41815 was elected leader, sending a full tablet report...
W20260811 01:44:58.202380 30156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:58.202840 30156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:58.203060 30156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:58.213892 30156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:44:58.214166 30156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:58.214346 30156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:58.214510 30156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:44:58.214594 30156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:44:58.229380 30156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:45607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:41815,127.21.214.125:41479,127.21.214.124:39589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.124.keytab
--principal=oryx/127.21.214.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.124:39589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.214.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:58.231398 30156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:58.232250 30264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.224259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45684 (local address 127.21.214.125:41479)
0811 01:44:58.225234 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:58.225239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:58.225255 (+ 16us) server_negotiation.cc:407] Connection header received
0811 01:44:58.225326 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:58.225329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:58.225431 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:58.225575 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:58.226359 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.227858 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.229960 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.231002 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.231213 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.231426 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:58.231695 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 01:44:58.231840 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":534,"threads_started":1}
I20260811 01:44:58.232292 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.224005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:41479 (local address 127.0.0.1:45684)
0811 01:44:58.224709 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:58.224756 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:58.224955 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:58.225641 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:58.225645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:58.225714 (+ 69us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:58.226099 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:58.226111 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.228066 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.228070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:58.229801 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:58.229815 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.231235 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.231241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:58.231462 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.231537 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 01:44:58.231665 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 01:44:58.231831 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":464,"threads_started":1}
I20260811 01:44:58.233418 30156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 01:44:58.233943 30212 raft_consensus.cc:1203] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260811 01:44:58.234817 30246 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 662c631857f64f2c96f1a494f2e6c01d. Latest consensus state: current_term: 1 leader_uuid: "662c631857f64f2c96f1a494f2e6c01d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: NON_VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: true } } }
I20260811 01:44:58.235177 30246 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [sys.catalog]: This master's current role is: LEARNER
W20260811 01:44:58.251652 30266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:58.252045 30267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:58.252687 30269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:58.254204 30156 server_base.cc:1034] running on GCE node
I20260811 01:44:58.255352 30156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:58.257360 30156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:58.258749 30156 hybrid_clock.cc:648] HybridClock initialized: now 1786412698258551 us; error 70 us; skew 500 ppm
I20260811 01:44:58.259234 30156 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:58 dist-test-slave-9gcw krb5kdc[29460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412698, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:58.264070 30156 init.cc:377] Logged in from keytab as oryx/127.21.214.124@KRBTEST.COM (short username oryx)
I20260811 01:44:58.266242 30156 webserver.cc:460] Webserver started at http://127.21.214.124:44035/ using document root <none> and password file <none>
I20260811 01:44:58.267107 30156 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:58.267225 30156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:58.267498 30156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:44:58.271549 30156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/data/instance:
uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d"
format_stamp: "Formatted at 2026-08-11 01:44:58 on dist-test-slave-9gcw"
server_key: "671f988c1718dab05a2a3745aa34930d"
server_key_iv: "71e82be9584420cae945c41a18099012"
server_key_version: "encryptionkey@0"
I20260811 01:44:58.272739 30156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/wal/instance:
uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d"
format_stamp: "Formatted at 2026-08-11 01:44:58 on dist-test-slave-9gcw"
server_key: "671f988c1718dab05a2a3745aa34930d"
server_key_iv: "71e82be9584420cae945c41a18099012"
server_key_version: "encryptionkey@0"
I20260811 01:44:58.279211 30156 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260811 01:44:58.283872 30278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:58.285586 30156 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 01:44:58.285905 30156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/wal
uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d"
format_stamp: "Formatted at 2026-08-11 01:44:58 on dist-test-slave-9gcw"
server_key: "671f988c1718dab05a2a3745aa34930d"
server_key_iv: "71e82be9584420cae945c41a18099012"
server_key_version: "encryptionkey@0"
I20260811 01:44:58.286217 30156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:58.303821 30156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:58.309693 30156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:58.310107 30156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:58.348416 30156 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.124:39589
I20260811 01:44:58.348428 30345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.124:39589 every 8 connection(s)
I20260811 01:44:58.351029 30156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/data/info.pb
I20260811 01:44:58.356199 30346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:44:58.359741 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 30156
I20260811 01:44:58.368973 30346 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:44:58 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.214.124@KRBTEST.COM
Aug 11 01:44:58 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412698, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.124@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM
I20260811 01:44:58.391371 30351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.362561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40132 (local address 127.21.214.124:39589)
0811 01:44:58.364081 (+ 1520us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:58.364099 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:58.364154 (+ 55us) server_negotiation.cc:407] Connection header received
0811 01:44:58.366550 (+ 2396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:58.366580 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:58.367081 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:58.367530 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:58.369862 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.371561 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.374272 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.375096 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.375370 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.380943 (+ 5573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:58.380954 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:58.380969 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:58.381034 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:58.385581 (+ 4547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.386359 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.386366 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.386374 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.386559 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.387127 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.387130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.387133 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.387891 (+ 758us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:58.388275 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:58.388895 (+ 620us) server_negotiation.cc:299] Negotiation successful
0811 01:44:58.389403 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1183,"thread_start_us":838,"threads_started":1}
I20260811 01:44:58.393944 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.372112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38798 (local address 127.21.214.126:41815)
0811 01:44:58.372707 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:58.372711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:58.373926 (+ 1215us) server_negotiation.cc:407] Connection header received
0811 01:44:58.374629 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:58.374636 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:58.374768 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:58.374906 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:58.376706 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.377952 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.379729 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.380340 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.380537 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.385671 (+ 5134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:58.385678 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:58.385683 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:58.385728 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:58.389782 (+ 4054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.390936 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.390949 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.390952 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.391287 (+ 335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.391867 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.391873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.391877 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.392329 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:58.392514 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:58.393306 (+ 792us) server_negotiation.cc:299] Negotiation successful
0811 01:44:58.393490 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":327,"threads_started":1}
I20260811 01:44:58.394539 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.372307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:38798)
0811 01:44:58.373708 (+ 1401us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:58.373749 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:58.374098 (+ 349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:58.374971 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:58.374980 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:58.376172 (+ 1192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:58.376516 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:58.376525 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.378139 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.378143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:58.379564 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:58.379577 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.380792 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.380804 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:58.380986 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.381888 (+ 902us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:58.381905 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:58.385406 (+ 3501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:58.389933 (+ 4527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:58.389954 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:58.389964 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:58.390789 (+ 825us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:58.391421 (+ 632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:58.391427 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:58.391431 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:58.391751 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:58.392503 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:58.392512 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:58.392821 (+ 309us) client_negotiation.cc:769] Sending connection context
0811 01:44:58.393185 (+ 364us) client_negotiation.cc:240] Negotiation successful
0811 01:44:58.393598 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1039,"thread_start_us":761,"threads_started":1}
I20260811 01:44:58.396302 30346 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:44:58 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412698, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.124@KRBTEST.COM for oryx/127.21.214.125@KRBTEST.COM
I20260811 01:44:58.412155 30264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.397497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45698 (local address 127.21.214.125:41479)
0811 01:44:58.397702 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:58.397706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:58.397726 (+ 20us) server_negotiation.cc:407] Connection header received
0811 01:44:58.397871 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:58.397874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:58.397935 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:58.398015 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:58.399062 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.400053 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.401640 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.402224 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.402333 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.406006 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:58.406010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:58.406014 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:58.406046 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:58.409430 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.410126 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.410130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.410132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.410304 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.410690 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.410695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.410698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.411069 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:58.411189 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:58.411585 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 01:44:58.411848 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 01:44:58.412165 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.397387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:41479 (local address 127.0.0.1:45698)
0811 01:44:58.397653 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:58.397671 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:58.397762 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:58.398086 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:58.398090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:58.398593 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:58.398929 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:58.398938 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.400283 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.400289 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:58.401491 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:58.401504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.402363 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.402368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:58.402528 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.403241 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:58.403267 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:58.405824 (+ 2557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:58.409568 (+ 3744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:58.409572 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:58.409574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:58.410009 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:58.410408 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:58.410411 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:58.410413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:58.410548 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:58.411193 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:58.411198 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:58.411330 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 01:44:58.411443 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 01:44:58.411686 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 01:44:58.413527 30346 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:44:58 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412698, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.124@KRBTEST.COM for oryx/127.21.214.124@KRBTEST.COM
I20260811 01:44:58.429386 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.414326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:39589 (local address 127.0.0.1:40144)
0811 01:44:58.414608 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:58.414624 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:58.414760 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:58.415061 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:58.415065 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:58.415586 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:58.415828 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:58.415838 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.416956 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.416964 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:58.418203 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:58.418216 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.418927 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.418932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:58.419332 (+ 400us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.419997 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:58.420015 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:58.422557 (+ 2542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:58.426352 (+ 3795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:58.426357 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:58.426362 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:58.426867 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:58.427311 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:58.427318 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:58.427324 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:58.427537 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:58.428200 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:58.428205 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:58.428381 (+ 176us) client_negotiation.cc:769] Sending connection context
0811 01:44:58.428525 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 01:44:58.428847 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 01:44:58.429450 30351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.414321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40144 (local address 127.21.214.124:39589)
0811 01:44:58.414544 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:58.414551 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:58.414692 (+ 141us) server_negotiation.cc:407] Connection header received
0811 01:44:58.414884 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:58.414895 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:58.414948 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:58.415045 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:58.415972 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.416827 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.418351 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.418763 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.418905 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.422785 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:58.422790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:58.422794 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:58.422824 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:58.426169 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.427000 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.427006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.427013 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.427176 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.427654 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.427661 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.427664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.428077 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:58.428239 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:58.428662 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 01:44:58.429040 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":113,"server-negotiator.queue_time_us":66}
I20260811 01:44:58.435999 30346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: Bootstrap starting.
I20260811 01:44:58.440547 30346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: Neither blocks nor log segments found. Creating new log.
I20260811 01:44:58.442605 30346 log.cc:824] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: Log is configured to *not* fsync() on all Append() calls
I20260811 01:44:58.447729 30346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: No bootstrap required, opened a new log
I20260811 01:44:58.461115 30346 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } }
I20260811 01:44:58.461634 30346 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:44:58.461835 30346 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22b5c1f3bba6482d8e5d02f5b8b2497d, State: Initialized, Role: FOLLOWER
I20260811 01:44:58.462520 30346 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } }
I20260811 01:44:58.463909 30355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } } }
I20260811 01:44:58.464592 30355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:44:58.465251 30346 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [sys.catalog]: configured and running, proceeding with master startup.
W20260811 01:44:58.478794 30368 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 01:44:58.479008 30368 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 01:44:58.484181 30156 master_runner.cc:428] Detected that this master 22b5c1f3bba6482d8e5d02f5b8b2497d is joining an existing cluster
I20260811 01:44:58.484375 30156 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.214.124:39589
I20260811 01:44:58.499572 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 29582
Aug 11 01:44:58 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.214.65@KRBTEST.COM
I20260811 01:44:58.534678 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.491989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:39589 (local address 127.0.0.1:40148)
0811 01:44:58.504692 (+ 12703us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:58.504723 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:58.505944 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:58.508021 (+ 2077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:58.508029 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:58.514867 (+ 6838us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:58.515883 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:58.515905 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.517886 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.517891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:58.520216 (+ 2325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:58.520228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.522300 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.522308 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:58.522511 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.523348 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:58.523379 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:58.525357 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:58.530550 (+ 5193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:58.530555 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:58.530559 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:58.531282 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:58.531966 (+ 684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:58.531971 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:58.531974 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:58.532202 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:58.533213 (+ 1011us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:58.533219 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:58.533477 (+ 258us) client_negotiation.cc:769] Sending connection context
0811 01:44:58.533637 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 01:44:58.534034 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12429,"mutex_wait_us":7,"thread_start_us":12084,"threads_started":1}
I20260811 01:44:58.535966 30351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.491061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40148 (local address 127.21.214.124:39589)
0811 01:44:58.491308 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:58.491316 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:58.504931 (+ 13615us) server_negotiation.cc:407] Connection header received
0811 01:44:58.507536 (+ 2605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:58.507543 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:58.507636 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:58.507755 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:58.516081 (+ 8326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.517547 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.520382 (+ 2835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.521741 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.521909 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.525536 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:58.525542 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:58.525547 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:58.525586 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:58.530386 (+ 4800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.531420 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.531425 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.531429 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.531815 (+ 386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.532359 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.532365 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.532368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.532935 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:58.533109 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:58.535074 (+ 1965us) server_negotiation.cc:299] Negotiation successful
0811 01:44:58.535405 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":87}
I20260811 01:44:58.537654 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.490222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38806 (local address 127.21.214.126:41815)
0811 01:44:58.490550 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:58.490554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:58.506353 (+ 15799us) server_negotiation.cc:407] Connection header received
0811 01:44:58.506753 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:58.506759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:58.506845 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:58.506918 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:58.516941 (+ 10023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.518830 (+ 1889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.522184 (+ 3354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.523219 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.523366 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.527114 (+ 3748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:58.527130 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:58.527135 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:58.527185 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:58.532898 (+ 5713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.533991 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.533998 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.534002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.534290 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.534985 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.534991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.534995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.535663 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:58.535865 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:58.536754 (+ 889us) server_negotiation.cc:299] Negotiation successful
0811 01:44:58.537063 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219}
I20260811 01:44:58.539037 30378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.492147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:38806)
0811 01:44:58.506207 (+ 14060us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:58.506228 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:58.506584 (+ 356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:58.507177 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:58.507182 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:58.515062 (+ 7880us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:58.516684 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:58.516706 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.519057 (+ 2351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.519063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:58.521898 (+ 2835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:58.521919 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.523799 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.523809 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:58.524194 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.525059 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:58.525088 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:58.526764 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:58.533105 (+ 6341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:58.533138 (+ 33us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:58.533161 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:58.533828 (+ 667us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:58.534478 (+ 650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:58.534483 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:58.534487 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:58.534829 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:58.535979 (+ 1150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:58.535986 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:58.536360 (+ 374us) client_negotiation.cc:769] Sending connection context
0811 01:44:58.537951 (+ 1591us) client_negotiation.cc:240] Negotiation successful
0811 01:44:58.538315 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13862,"mutex_wait_us":484,"spinlock_wait_cycles":98560,"thread_start_us":15247,"threads_started":1}
I20260811 01:44:58.539798 30264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.490222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45704 (local address 127.21.214.125:41479)
0811 01:44:58.490414 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:58.490420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:58.506348 (+ 15928us) server_negotiation.cc:407] Connection header received
0811 01:44:58.506710 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:58.506716 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:58.506820 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:58.506899 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:58.521171 (+ 14272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.522661 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.525519 (+ 2858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.526388 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.526534 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.529742 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:58.529751 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:58.529759 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:58.529807 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:58.534165 (+ 4358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.535590 (+ 1425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.535596 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.535600 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.535773 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.536693 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.536698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.536701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.537130 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:58.537315 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:58.538763 (+ 1448us) server_negotiation.cc:299] Negotiation successful
0811 01:44:58.539084 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 01:44:58.541702 30377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.492021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:41479 (local address 127.0.0.1:45704)
0811 01:44:58.506207 (+ 14186us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:58.506228 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:58.506506 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:58.507177 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:58.507182 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:58.515354 (+ 8172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:58.520797 (+ 5443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:58.520811 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.522926 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.522930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:58.525350 (+ 2420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:58.525360 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.526890 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.526898 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:58.527147 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.527945 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:58.527967 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:58.529356 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:58.534345 (+ 4989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:58.534350 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:58.534354 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:58.535433 (+ 1079us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:58.535986 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:58.535997 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:58.536001 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:58.536536 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:58.537403 (+ 867us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:58.537408 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:58.537660 (+ 252us) client_negotiation.cc:769] Sending connection context
0811 01:44:58.540366 (+ 2706us) client_negotiation.cc:240] Negotiation successful
0811 01:44:58.540862 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13855,"mutex_wait_us":321,"thread_start_us":13429,"threads_started":1}
I20260811 01:44:58.549998 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.501873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60916 (local address 127.21.214.65:34917)
0811 01:44:58.509986 (+ 8113us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:58.509997 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:58.510031 (+ 34us) server_negotiation.cc:407] Connection header received
0811 01:44:58.510104 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:58.510111 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:58.510338 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:58.510501 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:58.512966 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.515112 (+ 2146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.517445 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.518839 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.519860 (+ 1021us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.538426 (+ 18566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:58.538451 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:58.538475 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:58.538542 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:58.544862 (+ 6320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.545879 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.545892 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.545895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.546228 (+ 333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.547086 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.547092 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.547095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.547713 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:58.548167 (+ 454us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:58.548705 (+ 538us) server_negotiation.cc:299] Negotiation successful
0811 01:44:58.549099 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7901,"thread_start_us":7625,"threads_started":1}
I20260811 01:44:58.551756 29717 generic_service.cc:196] Checking for leaks (request via RPC)
Aug 11 01:44:58 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412695, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.126@KRBTEST.COM for oryx/127.21.214.124@KRBTEST.COM
W20260811 01:44:58.562916 30286 tablet.cc:2369] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:44:58.568493 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.546594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:39589 (local address 127.0.0.1:40154)
0811 01:44:58.547012 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:58.547035 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:58.547167 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:58.547799 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:58.547805 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:58.548528 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:58.548792 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:58.548807 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.550972 (+ 2165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.550978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:58.552226 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:58.552242 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.553420 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.553431 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:58.553703 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.554621 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:58.554648 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:58.558245 (+ 3597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:58.564155 (+ 5910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:58.564163 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:58.564168 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:58.565073 (+ 905us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:58.565734 (+ 661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:58.565740 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:58.565744 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:58.566013 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:58.566990 (+ 977us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:58.566996 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:58.567289 (+ 293us) client_negotiation.cc:769] Sending connection context
0811 01:44:58.567526 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 01:44:58.567879 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251}
I20260811 01:44:58.569028 30351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.546594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40154 (local address 127.21.214.124:39589)
0811 01:44:58.546820 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:58.546827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:58.547313 (+ 486us) server_negotiation.cc:407] Connection header received
0811 01:44:58.547423 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:58.547428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:58.547587 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:58.548971 (+ 1384us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:58.549277 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.550795 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.552387 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.553131 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.553301 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.558513 (+ 5212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:58.558521 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:58.558527 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:58.558575 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:58.563910 (+ 5335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.565298 (+ 1388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.565311 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.565316 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.565586 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.566253 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.566260 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.566265 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.566821 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:58.566995 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:58.567697 (+ 702us) server_negotiation.cc:299] Negotiation successful
0811 01:44:58.568001 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 01:44:58.571126 29510 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.214.124:39589: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d"
member_type: NON_VOTER
last_known_addr {
host: "127.21.214.124"
port: 39589
}
attrs {
promote: true
}
}
dest_uuid: "662c631857f64f2c96f1a494f2e6c01d"
cas_config_opid_index: 5
I20260811 01:44:58.572985 29510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: NON_VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: true } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: NON_VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: true } }
I20260811 01:44:58.575320 30262 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 22b5c1f3bba6482d8e5d02f5b8b2497d (127.21.214.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: NON_VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: true } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: NON_VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: true } } }
I20260811 01:44:58.589903 30317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:44:58.590057 30262 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.214.124:39589
I20260811 01:44:58.590909 30156 master.cc:537] Master@127.21.214.124:39589 shutting down...
I20260811 01:44:58.593840 30383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "662c631857f64f2c96f1a494f2e6c01d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: NON_VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: true } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: NON_VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: true } } }
I20260811 01:44:58.594416 30212 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 1 LEARNER]: Refusing update from remote peer 662c631857f64f2c96f1a494f2e6c01d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 01:44:58.594609 30383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:58.595094 30317 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 1 FOLLOWER]: Refusing update from remote peer 662c631857f64f2c96f1a494f2e6c01d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 01:44:58.595191 30382 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: NON_VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 01:44:58.596145 30262 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [LEADER]: Connected to new peer: Peer: permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: NON_VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:44:58.615211 30212 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 22b5c1f3bba6482d8e5d02f5b8b2497d (127.21.214.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: NON_VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: true } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: NON_VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: true } } }
I20260811 01:44:58.620210 30246 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "662c631857f64f2c96f1a494f2e6c01d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: NON_VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: true } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: NON_VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: true } } }
I20260811 01:44:58.620517 30246 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [sys.catalog]: This master's current role is: LEARNER
I20260811 01:44:58.621713 30262 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d: attempting to promote NON_VOTER bbd0420e2f7342008f02ce52a2778fbf to VOTER
I20260811 01:44:58.622609 30262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: NON_VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: true } }
I20260811 01:44:58.625484 30211 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 1 LEARNER]: Refusing update from remote peer 662c631857f64f2c96f1a494f2e6c01d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 01:44:58.626140 30382 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260811 01:44:58.628683 29494 consensus_peers.cc:487] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d -> Peer 22b5c1f3bba6482d8e5d02f5b8b2497d (127.21.214.124:39589): Couldn't send request to peer 22b5c1f3bba6482d8e5d02f5b8b2497d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 01:44:58.629014 30382 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bbd0420e2f7342008f02ce52a2778fbf (127.21.214.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: NON_VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: true } } }
I20260811 01:44:58.633677 30211 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bbd0420e2f7342008f02ce52a2778fbf (127.21.214.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: NON_VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: true } } }
I20260811 01:44:58.634820 30383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "662c631857f64f2c96f1a494f2e6c01d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: NON_VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: true } } }
I20260811 01:44:58.635138 30383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:58.635576 30383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "662c631857f64f2c96f1a494f2e6c01d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: NON_VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: true } } }
I20260811 01:44:58.635915 30383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:58.637954 30246 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "662c631857f64f2c96f1a494f2e6c01d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: NON_VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: true } } }
I20260811 01:44:58.638201 30246 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:44:58.639973 30156 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:44:58.640547 30156 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:44:58.640781 30156 tablet_replica.cc:330] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: stopping tablet replica
I20260811 01:44:58.659862 30156 master.cc:552] Master@127.21.214.124:39589 shutdown complete.
I20260811 01:44:58.659991 30156 master_runner.cc:305] Clearing existing system tablet
I20260811 01:44:58.660566 30156 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:58.660703 30156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:58.664179 30156 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 01:44:58.666577 30389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:58.667162 30156 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:58.667443 30156 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 01:44:58.667673 30156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/wal
uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d"
format_stamp: "Formatted at 2026-08-11 01:44:58 on dist-test-slave-9gcw"
server_key: "671f988c1718dab05a2a3745aa34930d"
server_key_iv: "71e82be9584420cae945c41a18099012"
server_key_version: "encryptionkey@0"
I20260811 01:44:58.673949 30156 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:44:58.683947 30156 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 01:44:58.684294 30156 log.cc:1196] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 01:44:58.685215 30156 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: Deleting consensus metadata
I20260811 01:44:58.685520 30156 master_runner.cc:315] Copying system tablet from 127.21.214.126:41815
I20260811 01:44:58.686415 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 29582
I20260811 01:44:58.701081 30156 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: tablet copy: Beginning tablet copy session from remote peer at address 127.21.214.126:41815
I20260811 01:44:58.716252 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:34917
--local_ip_for_outbound_sockets=127.21.214.65
--tserver_master_addrs=127.21.214.126:41815
--webserver_port=35537
--webserver_interface=127.21.214.65
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.65.keytab
--principal=oryx/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:45607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.214.126:41815,127.21.214.125:41479
--tserver_master_addrs=127.21.214.126:41815,127.21.214.125:41479,127.21.214.124:39589 with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:44:58.722896 30397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.702145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:38816)
0811 01:44:58.705605 (+ 3460us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:58.705632 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:58.705823 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:58.706299 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:58.706304 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:58.706856 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:58.707146 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:58.707170 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.709016 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.709022 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:58.710779 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:58.710795 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.712401 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.712413 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:58.712712 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.713616 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:58.713641 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:58.714832 (+ 1191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:58.719549 (+ 4717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:58.719555 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:58.719560 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:58.720098 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:58.720657 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:58.720662 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:58.720666 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:58.720838 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:58.721685 (+ 847us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:58.721692 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:58.721898 (+ 206us) client_negotiation.cc:769] Sending connection context
0811 01:44:58.722053 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 01:44:58.722429 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3307,"thread_start_us":3157,"threads_started":1}
I20260811 01:44:58.723129 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:58.702160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38816 (local address 127.21.214.126:41815)
0811 01:44:58.702411 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:58.702419 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:58.705769 (+ 3350us) server_negotiation.cc:407] Connection header received
0811 01:44:58.705997 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:58.706006 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:58.706115 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:58.706250 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:58.707361 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.708832 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.710953 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:58.712155 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:58.712331 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:58.715120 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:58.715128 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:58.715134 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:58.715182 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:58.719379 (+ 4197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.720308 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.720318 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.720324 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.720505 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:58.720978 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:58.720986 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:58.720992 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:58.721540 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:58.721717 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:58.722346 (+ 629us) server_negotiation.cc:299] Negotiation successful
0811 01:44:58.722644 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260811 01:44:58.724175 29536 tablet_copy_service.cc:140] P 662c631857f64f2c96f1a494f2e6c01d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 22b5c1f3bba6482d8e5d02f5b8b2497d ({username='oryx', principal='oryx/127.21.214.124@KRBTEST.COM'} at 127.0.0.1:38816)
I20260811 01:44:58.724385 29536 tablet_copy_service.cc:161] P 662c631857f64f2c96f1a494f2e6c01d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 22b5c1f3bba6482d8e5d02f5b8b2497d at {username='oryx', principal='oryx/127.21.214.124@KRBTEST.COM'} at 127.0.0.1:38816: session id = 22b5c1f3bba6482d8e5d02f5b8b2497d-00000000000000000000000000000000
I20260811 01:44:58.727666 29536 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 01:44:58.729928 30156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 01:44:58.738251 30156 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: tablet copy: Starting download of 0 data blocks...
I20260811 01:44:58.738639 30156 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: tablet copy: Starting download of 1 WAL segments...
I20260811 01:44:58.741803 30156 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 01:44:58.746752 29536 tablet_copy_service.cc:342] P 662c631857f64f2c96f1a494f2e6c01d: Request end of tablet copy session 22b5c1f3bba6482d8e5d02f5b8b2497d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.214.124@KRBTEST.COM'} at 127.0.0.1:38816
I20260811 01:44:58.746927 29536 tablet_copy_service.cc:433] P 662c631857f64f2c96f1a494f2e6c01d: ending tablet copy session 22b5c1f3bba6482d8e5d02f5b8b2497d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 22b5c1f3bba6482d8e5d02f5b8b2497d
I20260811 01:44:58.759902 30156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:58.762849 30400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:58.769356 30403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:58.774569 30405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:58.775107 30156 server_base.cc:1034] running on GCE node
I20260811 01:44:58.775465 30156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:58.776180 30156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:58.777386 30156 hybrid_clock.cc:648] HybridClock initialized: now 1786412698777338 us; error 38 us; skew 500 ppm
I20260811 01:44:58.777650 30156 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:58 dist-test-slave-9gcw krb5kdc[29460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412698, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:58.779768 30156 init.cc:377] Logged in from keytab as oryx/127.21.214.124@KRBTEST.COM (short username oryx)
I20260811 01:44:58.790421 30156 webserver.cc:460] Webserver started at http://127.21.214.124:41447/ using document root <none> and password file <none>
I20260811 01:44:58.790804 30156 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:58.790980 30156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:58.793805 30156 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 01:44:58.795611 30414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:58.796099 30156 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260811 01:44:58.796236 30156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/wal
uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d"
format_stamp: "Formatted at 2026-08-11 01:44:58 on dist-test-slave-9gcw"
server_key: "671f988c1718dab05a2a3745aa34930d"
server_key_iv: "71e82be9584420cae945c41a18099012"
server_key_version: "encryptionkey@0"
I20260811 01:44:58.796442 30156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:58.804174 30156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:58.808368 30156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:58.836604 30156 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.124:39589
I20260811 01:44:58.836627 30477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.124:39589 every 8 connection(s)
I20260811 01:44:58.837643 30156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/master-2/data/info.pb
I20260811 01:44:58.841545 30478 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:44:58.844393 30478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: Bootstrap starting.
I20260811 01:44:58.861370 30478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:44:58.862048 30478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: Bootstrap complete.
I20260811 01:44:58.863241 30478 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: NON_VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: true } }
I20260811 01:44:58.863673 30478 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 22b5c1f3bba6482d8e5d02f5b8b2497d, State: Initialized, Role: LEARNER
I20260811 01:44:58.863966 30478 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: NON_VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: true } }
I20260811 01:44:58.879923 30482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: NON_VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: true } } }
I20260811 01:44:58.880358 30478 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:44:58.880481 30482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [sys.catalog]: This master's current role is: LEARNER
I20260811 01:44:58.906035 30495 catalog_manager.cc:1259] Loaded cluster ID: 61662ac4f271493a9ff65577e0e6c353
I20260811 01:44:58.906175 30495 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: loading cluster ID for follower catalog manager: success
I20260811 01:44:58.908809 30495 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: acquiring CA information for follower catalog manager: success
I20260811 01:44:58.910847 30495 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:44:59.075935 30496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:59.066120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.21.214.124:39589)
0811 01:44:59.067514 (+ 1394us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:59.067520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:59.067537 (+ 17us) server_negotiation.cc:407] Connection header received
0811 01:44:59.067602 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:59.067606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:59.067675 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:59.067790 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:44:59.068411 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:59.070183 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:59.073518 (+ 3335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:59.074634 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:59.074829 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:59.074979 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:59.075449 (+ 470us) server_negotiation.cc:299] Negotiation successful
0811 01:44:59.075568 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1290,"thread_start_us":1049,"threads_started":1}
I20260811 01:44:59.076064 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:59.066083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:39589 (local address 127.0.0.1:40156)
0811 01:44:59.066352 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:59.066380 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:59.066635 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:59.067899 (+ 1264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:59.067906 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:59.067948 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:44:59.068174 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:59.068190 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:59.070413 (+ 2223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:59.070420 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:59.073258 (+ 2838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:59.073292 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:59.074909 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:59.074928 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:59.075224 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:59.075278 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 01:44:59.075430 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 01:44:59.075658 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 01:44:59.077409 30445 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 01:44:59.078097 30482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 662c631857f64f2c96f1a494f2e6c01d. Latest consensus state: current_term: 1 leader_uuid: "662c631857f64f2c96f1a494f2e6c01d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: NON_VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: true } } }
I20260811 01:44:59.078423 30482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [sys.catalog]: This master's current role is: LEARNER
W20260811 01:44:59.216192 30398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:59.216694 30398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:59.216962 30398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:59.227648 30398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:59.227975 30398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:59.228281 30398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:44:59.242748 30398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:45607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.65.keytab
--principal=oryx/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:34917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=35537
--tserver_master_addrs=127.21.214.126:41815,127.21.214.125:41479,127.21.214.124:39589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:44:59.244927 30398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:44:59.246752 30398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:44:59.264213 30501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:59.264516 30506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:44:59.265058 30503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:44:59.266845 30398 server_base.cc:1034] running on GCE node
I20260811 01:44:59.267614 30398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:44:59.269253 30398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:44:59.270529 30398 hybrid_clock.cc:648] HybridClock initialized: now 1786412699270472 us; error 39 us; skew 500 ppm
I20260811 01:44:59.271024 30398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:44:59 dist-test-slave-9gcw krb5kdc[29460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412699, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:44:59.276146 30398 init.cc:377] Logged in from keytab as oryx/127.21.214.65@KRBTEST.COM (short username oryx)
I20260811 01:44:59.278505 30398 webserver.cc:460] Webserver started at http://127.21.214.65:35537/ using document root <none> and password file <none>
I20260811 01:44:59.279350 30398 fs_manager.cc:359] Metadata directory not provided
I20260811 01:44:59.279531 30398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:44:59.287482 30398 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 01:44:59.291157 30514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:59.292824 30398 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 01:44:59.293038 30398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "26467dcaea844348b216edcec09ec9eb"
format_stamp: "Formatted at 2026-08-11 01:44:55 on dist-test-slave-9gcw"
server_key: "6ca4337a8478d6a3736b57a5e0ad116c"
server_key_iv: "fe74667a67cbe8a9873136b12116935d"
server_key_version: "encryptionkey@0"
I20260811 01:44:59.294305 30398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:44:59.327428 30398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:44:59.333452 30398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:44:59.333941 30398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:44:59.335433 30398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:44:59.337582 30398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:44:59.337777 30398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:59.337956 30398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:44:59.338165 30398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:44:59.411923 30398 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:34917
I20260811 01:44:59.411983 30643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:34917 every 8 connection(s)
I20260811 01:44:59.413792 30398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:44:59.418041 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 30398
I20260811 01:44:59.418586 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 29729
Aug 11 01:44:59 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.214.66@KRBTEST.COM
Aug 11 01:44:59 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412699, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.124@KRBTEST.COM
Aug 11 01:44:59 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412699, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM
Aug 11 01:44:59 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412699, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.125@KRBTEST.COM
I20260811 01:44:59.470077 30496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:59.420886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:41521 (local address 127.21.214.124:39589)
0811 01:44:59.421168 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:59.421174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:59.436442 (+ 15268us) server_negotiation.cc:407] Connection header received
0811 01:44:59.442042 (+ 5600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:59.442050 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:59.442148 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:59.442245 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:59.446729 (+ 4484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:59.447746 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:59.451031 (+ 3285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:59.451875 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:59.451994 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:59.459621 (+ 7627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:59.459631 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:59.459636 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:59.459689 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:59.464654 (+ 4965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:59.465816 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:59.465822 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:59.465825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:59.466037 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:59.466650 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:59.466655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:59.466658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:59.467075 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:59.467438 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:59.468940 (+ 1502us) server_negotiation.cc:299] Negotiation successful
0811 01:44:59.469411 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 01:44:59.471496 30650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:59.420010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52542 (local address 127.21.214.66:38921)
0811 01:44:59.436063 (+ 16053us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:59.436074 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:59.437279 (+ 1205us) server_negotiation.cc:407] Connection header received
0811 01:44:59.437381 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:59.437391 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:59.437846 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:59.438086 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:59.441624 (+ 3538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:59.443630 (+ 2006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:59.445898 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:59.447468 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:59.447586 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:59.458296 (+ 10710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:59.458314 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:59.458330 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:59.458410 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:59.464619 (+ 6209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:59.465845 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:59.465855 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:59.465858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:59.466106 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:59.466619 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:59.466627 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:59.466630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:59.467571 (+ 941us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:59.468325 (+ 754us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:59.470309 (+ 1984us) server_negotiation.cc:299] Negotiation successful
0811 01:44:59.470711 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15708,"thread_start_us":16037,"threads_started":1}
I20260811 01:44:59.471565 30652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:59.423012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:43715 (local address 127.21.214.126:41815)
0811 01:44:59.437425 (+ 14413us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:59.437432 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:59.437451 (+ 19us) server_negotiation.cc:407] Connection header received
0811 01:44:59.442134 (+ 4683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:59.442140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:59.442326 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:59.442456 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:59.447534 (+ 5078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:59.448822 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:59.451572 (+ 2750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:59.452351 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:59.452510 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:59.460178 (+ 7668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:59.460184 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:59.460187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:59.460231 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:59.467273 (+ 7042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:59.468814 (+ 1541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:59.468823 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:59.468827 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:59.469039 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:59.469710 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:59.469716 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:59.469719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:59.470206 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:59.470344 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:59.470851 (+ 507us) server_negotiation.cc:299] Negotiation successful
0811 01:44:59.471126 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14280,"thread_start_us":14020,"threads_started":1}
I20260811 01:44:59.473425 29870 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:44:59.473603 30654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:59.423101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.65:43715)
0811 01:44:59.434142 (+ 11041us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:59.434170 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:59.441656 (+ 7486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:59.442853 (+ 1197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:59.442866 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:59.445231 (+ 2365us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:59.447285 (+ 2054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:59.447303 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:59.448971 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:59.448976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:59.451301 (+ 2325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:59.451328 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:59.452858 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:59.452865 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:59.453118 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:59.454055 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:59.454094 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:59.459869 (+ 5775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:59.467520 (+ 7651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:59.467527 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:59.467531 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:59.468517 (+ 986us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:59.469210 (+ 693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:59.469215 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:59.469219 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:59.469545 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:59.470393 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:59.470397 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:59.470536 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 01:44:59.471856 (+ 1320us) client_negotiation.cc:240] Negotiation successful
0811 01:44:59.472294 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10817,"mutex_wait_us":442,"spinlock_wait_cycles":2432,"thread_start_us":10388,"threads_started":1}
I20260811 01:44:59.473716 30655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:59.423941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:54565 (local address 127.21.214.125:41479)
0811 01:44:59.435245 (+ 11304us) server_negotiation.cc:206] Beginning negotiation
0811 01:44:59.435252 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:44:59.435366 (+ 114us) server_negotiation.cc:407] Connection header received
0811 01:44:59.442151 (+ 6785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:44:59.442158 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:44:59.442338 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:44:59.442438 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:44:59.449081 (+ 6643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:59.450686 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:59.453353 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:44:59.454280 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:44:59.454406 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:59.462565 (+ 8159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:44:59.462574 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:44:59.462582 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:44:59.462639 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:44:59.467706 (+ 5067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:59.468856 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:59.468862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:59.468865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:59.469030 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:44:59.469710 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:44:59.469716 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:44:59.469719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:44:59.470147 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:44:59.470310 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 01:44:59.472641 (+ 2331us) server_negotiation.cc:299] Negotiation successful
0811 01:44:59.472917 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11118,"thread_start_us":10904,"threads_started":1}
I20260811 01:44:59.474313 30651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:59.421711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:41479 (local address 127.21.214.65:54565)
0811 01:44:59.429993 (+ 8282us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:59.430082 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:59.441640 (+ 11558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:59.443922 (+ 2282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:59.443927 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:59.445232 (+ 1305us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:59.446627 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:59.446642 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:59.450943 (+ 4301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:59.450949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:59.453122 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:59.453136 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:59.454920 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:59.454930 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:59.455268 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:59.457504 (+ 2236us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:59.457521 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:59.462182 (+ 4661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:59.468015 (+ 5833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:59.468020 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:59.468024 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:59.468707 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:59.469215 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:59.469219 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:59.469223 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:59.469579 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:59.470378 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:59.470382 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:59.470534 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 01:44:59.470730 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 01:44:59.471439 (+ 709us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7727,"mutex_wait_us":55,"spinlock_wait_cycles":4864,"thread_start_us":7426,"threads_started":1}
I20260811 01:44:59.474347 30656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:44:59.424914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:39589 (local address 127.21.214.65:41521)
0811 01:44:59.436300 (+ 11386us) negotiation.cc:107] Waiting for socket to connect
0811 01:44:59.436321 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:44:59.441656 (+ 5335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:44:59.442853 (+ 1197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:44:59.442866 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:44:59.445231 (+ 2365us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:44:59.446432 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:59.446458 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:59.447932 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:59.447936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:59.450766 (+ 2830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:44:59.450786 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:44:59.452446 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:44:59.452456 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:44:59.453021 (+ 565us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:44:59.454257 (+ 1236us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:44:59.454301 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:44:59.459003 (+ 4702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:44:59.464850 (+ 5847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:59.464861 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:59.464880 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:59.465654 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:59.466222 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:44:59.466229 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:44:59.466233 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:44:59.466515 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:44:59.467598 (+ 1083us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:44:59.467607 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:44:59.468339 (+ 732us) client_negotiation.cc:769] Sending connection context
0811 01:44:59.470838 (+ 2499us) client_negotiation.cc:240] Negotiation successful
0811 01:44:59.472122 (+ 1284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11231,"mutex_wait_us":771,"spinlock_wait_cycles":46336,"thread_start_us":10995,"threads_started":1}
I20260811 01:44:59.476284 30645 heartbeater.cc:347] Connected to a master server at 127.21.214.126:41815
I20260811 01:44:59.476341 30647 heartbeater.cc:347] Connected to a master server at 127.21.214.125:41479
I20260811 01:44:59.476573 30644 heartbeater.cc:347] Connected to a master server at 127.21.214.124:39589
I20260811 01:44:59.476871 30647 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:59.476871 30645 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:59.476897 30644 heartbeater.cc:464] Registering TS with master...
I20260811 01:44:59.478643 30644 heartbeater.cc:511] Master 127.21.214.124:39589 requested a full tablet report, sending...
I20260811 01:44:59.478581 30645 heartbeater.cc:511] Master 127.21.214.126:41815 requested a full tablet report, sending...
I20260811 01:44:59.478714 30647 heartbeater.cc:511] Master 127.21.214.125:41479 requested a full tablet report, sending...
I20260811 01:44:59.481135 29510 ts_manager.cc:194] Re-registered known tserver with Master: 26467dcaea844348b216edcec09ec9eb (127.21.214.65:34917)
I20260811 01:44:59.481954 30200 ts_manager.cc:194] Registered new tserver with Master: 26467dcaea844348b216edcec09ec9eb (127.21.214.65:34917)
I20260811 01:44:59.482462 29510 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:43715
I20260811 01:44:59.482648 30433 ts_manager.cc:194] Registered new tserver with Master: 26467dcaea844348b216edcec09ec9eb (127.21.214.65:34917)
I20260811 01:44:59.562693 30382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "662c631857f64f2c96f1a494f2e6c01d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: NON_VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: true } } }
I20260811 01:44:59.563287 30382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:59.565400 30382 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d: attempting to promote NON_VOTER 22b5c1f3bba6482d8e5d02f5b8b2497d to VOTER
I20260811 01:44:59.566233 30382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } }
I20260811 01:44:59.568436 30445 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 1 LEARNER]: Refusing update from remote peer 662c631857f64f2c96f1a494f2e6c01d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 01:44:59.568573 30211 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 1 FOLLOWER]: Refusing update from remote peer 662c631857f64f2c96f1a494f2e6c01d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 01:44:59.569329 30382 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [LEADER]: Connected to new peer: Peer: permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 01:44:59.569788 30659 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 01:44:59.573573 30659 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 22b5c1f3bba6482d8e5d02f5b8b2497d (127.21.214.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } } }
I20260811 01:44:59.574000 30211 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 22b5c1f3bba6482d8e5d02f5b8b2497d (127.21.214.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } } }
I20260811 01:44:59.580845 30658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "662c631857f64f2c96f1a494f2e6c01d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } } }
I20260811 01:44:59.580940 30382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "662c631857f64f2c96f1a494f2e6c01d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } } }
I20260811 01:44:59.581265 30658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:59.581283 30382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:59.583465 30382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "662c631857f64f2c96f1a494f2e6c01d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } } }
I20260811 01:44:59.583747 30382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d [sys.catalog]: This master's current role is: LEADER
I20260811 01:44:59.584764 30445 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 22b5c1f3bba6482d8e5d02f5b8b2497d (127.21.214.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } } }
I20260811 01:44:59.586335 30662 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "662c631857f64f2c96f1a494f2e6c01d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } } }
I20260811 01:44:59.587105 30662 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:44:59.590627 30663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "662c631857f64f2c96f1a494f2e6c01d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } } }
I20260811 01:44:59.592028 30663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:44:59.593590 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 29729
I20260811 01:44:59.620007 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:38921
--local_ip_for_outbound_sockets=127.21.214.66
--tserver_master_addrs=127.21.214.126:41815
--webserver_port=37873
--webserver_interface=127.21.214.66
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.66.keytab
--principal=oryx/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:45607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.214.126:41815,127.21.214.125:41479
--tserver_master_addrs=127.21.214.126:41815,127.21.214.125:41479,127.21.214.124:39589 with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:44:59.982638 30665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:44:59.982946 30665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:44:59.983103 30665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:44:59.990651 30665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:44:59.990780 30665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:44:59.990897 30665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:45:00.002017 30665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:45607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.66.keytab
--principal=oryx/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:38921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=37873
--tserver_master_addrs=127.21.214.126:41815,127.21.214.125:41479,127.21.214.124:39589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:00.003559 30665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:00.005007 30665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:00.020140 30673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:00.020031 30671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:00.020071 30670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:00.021440 30665 server_base.cc:1034] running on GCE node
I20260811 01:45:00.022161 30665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:00.023308 30665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:00.024565 30665 hybrid_clock.cc:648] HybridClock initialized: now 1786412700024517 us; error 52 us; skew 500 ppm
I20260811 01:45:00.024915 30665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:45:00 dist-test-slave-9gcw krb5kdc[29460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412700, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:45:00.028424 30665 init.cc:377] Logged in from keytab as oryx/127.21.214.66@KRBTEST.COM (short username oryx)
I20260811 01:45:00.030201 30665 webserver.cc:460] Webserver started at http://127.21.214.66:37873/ using document root <none> and password file <none>
I20260811 01:45:00.031044 30665 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:00.031214 30665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:00.037410 30665 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 01:45:00.040421 30680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:00.042114 30665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:45:00.042263 30665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "56802ed9c3784965b20f771a4840de0b"
format_stamp: "Formatted at 2026-08-11 01:44:56 on dist-test-slave-9gcw"
server_key: "a61a6f6c23ef3c1bf511648842a4e034"
server_key_iv: "b06031b05f3e76ddfce448e29126fc77"
server_key_version: "encryptionkey@0"
I20260811 01:45:00.043303 30665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:00.059792 30665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:00.065110 30665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:00.065470 30665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:00.066547 30665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:45:00.068078 30665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:00.068205 30665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:00.068320 30665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:00.068394 30665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:00.108209 30665 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:38921
I20260811 01:45:00.108264 30814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:38921 every 8 connection(s)
I20260811 01:45:00.109980 30665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:45:00.119585 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 30665
I20260811 01:45:00.119952 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 29884
Aug 11 01:45:00 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412700, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM
Aug 11 01:45:00 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412700, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.125@KRBTEST.COM
Aug 11 01:45:00 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412700, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.124@KRBTEST.COM
Aug 11 01:45:00 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.214.67@KRBTEST.COM
I20260811 01:45:00.173929 30820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.113511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:55447 (local address 127.21.214.125:41479)
0811 01:45:00.145010 (+ 31499us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:00.145019 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:00.145047 (+ 28us) server_negotiation.cc:407] Connection header received
0811 01:45:00.147076 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:00.147096 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:00.147232 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:00.147373 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:00.149997 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.151189 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.154855 (+ 3666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.155558 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.155695 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.163034 (+ 7339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:00.163041 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:00.163046 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:00.163106 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:00.168693 (+ 5587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.170062 (+ 1369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.170067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.170070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.170322 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.171096 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.171102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.171106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.171695 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:00.171845 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:00.173186 (+ 1341us) server_negotiation.cc:299] Negotiation successful
0811 01:45:00.173454 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31331,"thread_start_us":32297,"threads_started":1}
I20260811 01:45:00.174496 30821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.113886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:52055 (local address 127.21.214.126:41815)
0811 01:45:00.144347 (+ 30461us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:00.144357 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:00.144390 (+ 33us) server_negotiation.cc:407] Connection header received
0811 01:45:00.147754 (+ 3364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:00.147761 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:00.147853 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:00.147935 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:00.149981 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.151163 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.154935 (+ 3772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.155709 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.155815 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.162320 (+ 6505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:00.162327 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:00.162332 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:00.162377 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:00.167961 (+ 5584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.171492 (+ 3531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.171498 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.171500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.171744 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.172391 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.172397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.172401 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.172838 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:00.172981 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:00.173629 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 01:45:00.173900 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30277,"thread_start_us":30519,"threads_started":1}
I20260811 01:45:00.175537 30824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.115613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:41479 (local address 127.21.214.66:55447)
0811 01:45:00.142540 (+ 26927us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:00.142569 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:00.146509 (+ 3940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:00.147708 (+ 1199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:00.147721 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:00.148730 (+ 1009us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:00.149746 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:00.149766 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.153151 (+ 3385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.153156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:00.154650 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:00.154668 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.156105 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.156135 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:00.156591 (+ 456us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.157837 (+ 1246us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:00.157859 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:00.162678 (+ 4819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:00.168866 (+ 6188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:00.168871 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:00.168874 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:00.169920 (+ 1046us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:00.170501 (+ 581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:00.170513 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:00.170516 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:00.170932 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:00.171988 (+ 1056us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:00.171992 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:00.172599 (+ 607us) client_negotiation.cc:769] Sending connection context
0811 01:45:00.174269 (+ 1670us) client_negotiation.cc:240] Negotiation successful
0811 01:45:00.174568 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26773,"mutex_wait_us":135,"spinlock_wait_cycles":40704,"thread_start_us":26574,"threads_started":1}
I20260811 01:45:00.176057 30822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.114289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.66:52055)
0811 01:45:00.138532 (+ 24243us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:00.138612 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:00.147561 (+ 8949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:00.148206 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:00.148213 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:00.148763 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:00.149734 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:00.149763 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.151403 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.151409 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:00.154756 (+ 3347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:00.154775 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.156201 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.156210 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:00.156576 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.157793 (+ 1217us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:00.157840 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:00.161938 (+ 4098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:00.170683 (+ 8745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:00.170689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:00.170692 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:00.171349 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:00.171901 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:00.171905 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:00.171908 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:00.172228 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:00.173065 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:00.173069 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:00.173283 (+ 214us) client_negotiation.cc:769] Sending connection context
0811 01:45:00.174934 (+ 1651us) client_negotiation.cc:240] Negotiation successful
0811 01:45:00.175267 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23732,"mutex_wait_us":101,"spinlock_wait_cycles":26624,"thread_start_us":23388,"threads_started":1}
I20260811 01:45:00.176395 30819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.113316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:52249 (local address 127.21.214.124:39589)
0811 01:45:00.151518 (+ 38202us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:00.151524 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:00.151548 (+ 24us) server_negotiation.cc:407] Connection header received
0811 01:45:00.151602 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:00.151607 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:00.151675 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:00.151797 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:00.153137 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.154505 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.157107 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.158028 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.158152 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.163242 (+ 5090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:00.163260 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:00.163267 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:00.163313 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:00.168609 (+ 5296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.170056 (+ 1447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.170064 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.170069 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.170274 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.171164 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.171169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.171172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.171638 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:00.171834 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:00.175547 (+ 3713us) server_negotiation.cc:299] Negotiation successful
0811 01:45:00.175894 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38055,"thread_start_us":38592,"threads_started":1}
I20260811 01:45:00.176796 30823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.115050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:39589 (local address 127.21.214.66:52249)
0811 01:45:00.141802 (+ 26752us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:00.141832 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:00.146552 (+ 4720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:00.152225 (+ 5673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:00.152228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:00.152715 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:00.152972 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:00.152980 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.154709 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.154716 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:00.156886 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:00.156903 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.158555 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.158562 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:00.158883 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.159721 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:00.159755 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:00.162925 (+ 3170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:00.168824 (+ 5899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:00.168837 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:00.168862 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:00.169886 (+ 1024us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:00.170471 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:00.170478 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:00.170482 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:00.170981 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:00.171982 (+ 1001us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:00.171991 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:00.172608 (+ 617us) client_negotiation.cc:769] Sending connection context
0811 01:45:00.173049 (+ 441us) client_negotiation.cc:240] Negotiation successful
0811 01:45:00.173782 (+ 733us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26549,"mutex_wait_us":48,"spinlock_wait_cycles":4736,"thread_start_us":26306,"threads_started":1}
I20260811 01:45:00.177927 30817 heartbeater.cc:347] Connected to a master server at 127.21.214.125:41479
I20260811 01:45:00.178232 30815 heartbeater.cc:347] Connected to a master server at 127.21.214.124:39589
I20260811 01:45:00.178318 30817 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:00.178407 30815 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:00.179512 30815 heartbeater.cc:511] Master 127.21.214.124:39589 requested a full tablet report, sending...
I20260811 01:45:00.179661 30817 heartbeater.cc:511] Master 127.21.214.125:41479 requested a full tablet report, sending...
I20260811 01:45:00.180358 30816 heartbeater.cc:347] Connected to a master server at 127.21.214.126:41815
I20260811 01:45:00.180562 30816 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:00.181134 30816 heartbeater.cc:511] Master 127.21.214.126:41815 requested a full tablet report, sending...
I20260811 01:45:00.182132 30200 ts_manager.cc:194] Registered new tserver with Master: 56802ed9c3784965b20f771a4840de0b (127.21.214.66:38921)
I20260811 01:45:00.182286 30433 ts_manager.cc:194] Registered new tserver with Master: 56802ed9c3784965b20f771a4840de0b (127.21.214.66:38921)
I20260811 01:45:00.182386 29510 ts_manager.cc:194] Re-registered known tserver with Master: 56802ed9c3784965b20f771a4840de0b (127.21.214.66:38921)
I20260811 01:45:00.183465 29510 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:52055
I20260811 01:45:00.184876 30827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.121406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40314 (local address 127.21.214.67:38341)
0811 01:45:00.144700 (+ 23294us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:00.144716 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:00.151394 (+ 6678us) server_negotiation.cc:407] Connection header received
0811 01:45:00.151487 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:00.151496 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:00.151771 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:00.151944 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:00.156687 (+ 4743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.158669 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.163708 (+ 5039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.164801 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.164921 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.174493 (+ 9572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:00.174512 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:00.174531 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:00.174594 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:00.179691 (+ 5097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.181409 (+ 1718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.181420 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.181423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.181611 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.182217 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.182224 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.182226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.182803 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:00.183172 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:00.183954 (+ 782us) server_negotiation.cc:299] Negotiation successful
0811 01:45:00.184264 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22995,"thread_start_us":23309,"threads_started":1}
I20260811 01:45:00.186335 30026 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:00.279479 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 29884
I20260811 01:45:00.303654 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:38341
--local_ip_for_outbound_sockets=127.21.214.67
--tserver_master_addrs=127.21.214.126:41815
--webserver_port=44563
--webserver_interface=127.21.214.67
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.67.keytab
--principal=oryx/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:45607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.214.126:41815,127.21.214.125:41479
--tserver_master_addrs=127.21.214.126:41815,127.21.214.125:41479,127.21.214.124:39589 with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:45:00.485654 30645 heartbeater.cc:503] Master 127.21.214.126:41815 was elected leader, sending a full tablet report...
W20260811 01:45:00.668351 30829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:00.668838 30829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:00.669042 30829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:00.676800 30829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:00.676978 30829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:00.677248 30829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:45:00.688320 30829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:45607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.67.keytab
--principal=oryx/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:38341
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=44563
--tserver_master_addrs=127.21.214.126:41815,127.21.214.125:41479,127.21.214.124:39589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:00.689941 30829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:00.691506 30829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:00.706700 30834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:00.706756 30836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:00.707160 30838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:00.708197 30829 server_base.cc:1034] running on GCE node
I20260811 01:45:00.708972 30829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:00.710199 30829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:00.711467 30829 hybrid_clock.cc:648] HybridClock initialized: now 1786412700711390 us; error 67 us; skew 500 ppm
I20260811 01:45:00.711946 30829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:45:00 dist-test-slave-9gcw krb5kdc[29460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412700, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:45:00.719800 30829 init.cc:377] Logged in from keytab as oryx/127.21.214.67@KRBTEST.COM (short username oryx)
I20260811 01:45:00.721653 30829 webserver.cc:460] Webserver started at http://127.21.214.67:44563/ using document root <none> and password file <none>
I20260811 01:45:00.722503 30829 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:00.722704 30829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:00.729440 30829 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 01:45:00.732749 30846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:00.734512 30829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:45:00.734872 30829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "39a96547f8ec4bf08372b8ce43c159a7"
format_stamp: "Formatted at 2026-08-11 01:44:56 on dist-test-slave-9gcw"
server_key: "3e8c87ee95538c52979a42ef24da4bc8"
server_key_iv: "7a217b4762740511b5e802028d8596dd"
server_key_version: "encryptionkey@0"
I20260811 01:45:00.736070 30829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:00.760671 30829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:00.765830 30829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:00.766108 30829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:00.767192 30829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:45:00.768764 30829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:00.768980 30829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:00.769090 30829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:00.769208 30829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:00.806520 30829 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:38341
I20260811 01:45:00.806574 30973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:38341 every 8 connection(s)
I20260811 01:45:00.808094 30829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:45:00.812964 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 30829
Aug 11 01:45:00 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412700, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM
Aug 11 01:45:00 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412700, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.125@KRBTEST.COM
Aug 11 01:45:00 dist-test-slave-9gcw krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412700, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.124@KRBTEST.COM
I20260811 01:45:00.874922 30984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.815525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:41479 (local address 127.21.214.67:49067)
0811 01:45:00.831619 (+ 16094us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:00.831638 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:00.832947 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:00.834162 (+ 1215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:00.834175 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:00.835546 (+ 1371us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:00.836282 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:00.836321 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.837997 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.838003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:00.840061 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:00.840076 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.847412 (+ 7336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.847421 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:00.847615 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.850772 (+ 3157us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:00.850801 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:00.855941 (+ 5140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:00.862191 (+ 6250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:00.862202 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:00.862225 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:00.863001 (+ 776us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:00.869531 (+ 6530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:00.869535 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:00.869538 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:00.870070 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:00.871178 (+ 1108us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:00.871188 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:00.871777 (+ 589us) client_negotiation.cc:769] Sending connection context
0811 01:45:00.872221 (+ 444us) client_negotiation.cc:240] Negotiation successful
0811 01:45:00.873053 (+ 832us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15939,"thread_start_us":15044,"threads_started":1}
I20260811 01:45:00.876498 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.811776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:49067 (local address 127.21.214.125:41479)
0811 01:45:00.828478 (+ 16702us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:00.828482 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:00.831780 (+ 3298us) server_negotiation.cc:407] Connection header received
0811 01:45:00.833458 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:00.833464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:00.833557 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:00.833671 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:00.836544 (+ 2873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.837814 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.840254 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.840924 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.841024 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.856169 (+ 15145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:00.856176 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:00.856181 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:00.856224 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:00.861936 (+ 5712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.863161 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.863167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.863171 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.863381 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.870321 (+ 6940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.870328 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.870331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.870894 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:00.871081 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:00.875691 (+ 4610us) server_negotiation.cc:299] Negotiation successful
0811 01:45:00.876024 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16587,"thread_start_us":16437,"threads_started":1}
I20260811 01:45:00.878031 30976 heartbeater.cc:347] Connected to a master server at 127.21.214.125:41479
I20260811 01:45:00.878480 30976 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:00.879814 30978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.811693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:44401 (local address 127.21.214.126:41815)
0811 01:45:00.828045 (+ 16352us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:00.828056 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:00.831047 (+ 2991us) server_negotiation.cc:407] Connection header received
0811 01:45:00.833444 (+ 2397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:00.833450 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:00.833554 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:00.833667 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:00.841741 (+ 8074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.842903 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.844898 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.845604 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.845722 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.863690 (+ 17968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:00.863698 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:00.863704 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:00.863751 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:00.868114 (+ 4363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.869073 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.869082 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.869088 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.869356 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.871515 (+ 2159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.871520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.871523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.871937 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:00.872089 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:00.879261 (+ 7172us) server_negotiation.cc:299] Negotiation successful
0811 01:45:00.879490 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16091,"thread_start_us":16786,"threads_started":1}
I20260811 01:45:00.879899 30976 heartbeater.cc:511] Master 127.21.214.125:41479 requested a full tablet report, sending...
I20260811 01:45:00.882364 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.815235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.67:44401)
0811 01:45:00.830865 (+ 15630us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:00.830883 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:00.832916 (+ 2033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:00.834158 (+ 1242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:00.834173 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:00.835328 (+ 1155us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:00.841529 (+ 6201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:00.841542 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.843087 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.843093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:00.844720 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:00.844735 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.846244 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.846253 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:00.846564 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.848510 (+ 1946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:00.848549 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:00.854946 (+ 6397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:00.868329 (+ 13383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:00.868334 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:00.868338 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:00.868872 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:00.869505 (+ 633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:00.869511 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:00.869515 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:00.869904 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:00.877039 (+ 7135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:00.877047 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:00.877336 (+ 289us) client_negotiation.cc:769] Sending connection context
0811 01:45:00.877534 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 01:45:00.877897 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15498,"thread_start_us":16098,"threads_started":1}
I20260811 01:45:00.882675 30200 ts_manager.cc:194] Registered new tserver with Master: 39a96547f8ec4bf08372b8ce43c159a7 (127.21.214.67:38341)
I20260811 01:45:00.885558 30975 heartbeater.cc:347] Connected to a master server at 127.21.214.126:41815
I20260811 01:45:00.885612 30981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.813114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:50119 (local address 127.21.214.124:39589)
0811 01:45:00.829547 (+ 16433us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:00.829553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:00.830357 (+ 804us) server_negotiation.cc:407] Connection header received
0811 01:45:00.833498 (+ 3141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:00.833503 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:00.833585 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:00.833668 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:00.847550 (+ 13882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.848741 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.850894 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.851533 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.851635 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.874406 (+ 22771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:00.874415 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:00.874420 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:00.874469 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:00.878930 (+ 4461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.881107 (+ 2177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.881118 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.881124 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.881617 (+ 493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.883183 (+ 1566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.883192 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.883198 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.883689 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:00.883912 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:00.884777 (+ 865us) server_negotiation.cc:299] Negotiation successful
0811 01:45:00.885121 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16326,"thread_start_us":16216,"threads_started":1}
I20260811 01:45:00.885756 30975 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:00.886510 30975 heartbeater.cc:511] Master 127.21.214.126:41815 requested a full tablet report, sending...
I20260811 01:45:00.887405 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.812459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:39589 (local address 127.21.214.67:50119)
0811 01:45:00.830133 (+ 17674us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:00.830187 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:00.832916 (+ 2729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:00.834158 (+ 1242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:00.834173 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:00.835328 (+ 1155us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:00.847350 (+ 12022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:00.847363 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.848967 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.848974 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:00.850680 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:00.850701 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.851993 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.851999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:00.852300 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.853224 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:00.853410 (+ 186us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:00.874030 (+ 20620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:00.880321 (+ 6291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:00.880328 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:00.880334 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:00.880943 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:00.881869 (+ 926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:00.881878 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:00.881884 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:00.882987 (+ 1103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:00.884027 (+ 1040us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:00.884035 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:00.884254 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 01:45:00.886335 (+ 2081us) client_negotiation.cc:240] Negotiation successful
0811 01:45:00.886790 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17355,"spinlock_wait_cycles":19840,"thread_start_us":17125,"threads_started":1}
I20260811 01:45:00.887982 29510 ts_manager.cc:194] Re-registered known tserver with Master: 39a96547f8ec4bf08372b8ce43c159a7 (127.21.214.67:38341)
I20260811 01:45:00.889564 29510 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:44401
I20260811 01:45:00.890733 30974 heartbeater.cc:347] Connected to a master server at 127.21.214.124:39589
I20260811 01:45:00.890928 30974 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:00.891376 30974 heartbeater.cc:511] Master 127.21.214.124:39589 requested a full tablet report, sending...
I20260811 01:45:00.893666 30433 ts_manager.cc:194] Registered new tserver with Master: 39a96547f8ec4bf08372b8ce43c159a7 (127.21.214.67:38341)
I20260811 01:45:00.919661 30978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.853537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38818 (local address 127.21.214.126:41815)
0811 01:45:00.880197 (+ 26660us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:00.880203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:00.897309 (+ 17106us) server_negotiation.cc:407] Connection header received
0811 01:45:00.897510 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:00.897515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:00.897665 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:00.897798 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:00.902080 (+ 4282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.903065 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.905624 (+ 2559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.906314 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.906418 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.911128 (+ 4710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:00.911138 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:00.911146 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:00.911190 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:00.915705 (+ 4515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.916741 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.916747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.916752 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.916918 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.917456 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.917460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.917464 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.918018 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:00.918331 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:00.918918 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 01:45:00.919199 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26519,"thread_start_us":32105,"threads_started":1}
I20260811 01:45:00.920343 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.834620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45706 (local address 127.21.214.125:41479)
0811 01:45:00.878237 (+ 43617us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:00.878246 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:00.888163 (+ 9917us) server_negotiation.cc:407] Connection header received
0811 01:45:00.888572 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:00.888579 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:00.888688 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:00.888889 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:00.897089 (+ 8200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.898420 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.901304 (+ 2884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.901894 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.902052 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.910014 (+ 7962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:00.910025 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:00.910030 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:00.910110 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:00.914720 (+ 4610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.917453 (+ 2733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.917458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.917462 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.917678 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.918101 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.918107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.918110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.918531 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:00.918888 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:00.919284 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 01:45:00.919540 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43488,"thread_start_us":56056,"threads_started":1}
I20260811 01:45:00.920760 30981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.841676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40158 (local address 127.21.214.124:39589)
0811 01:45:00.886284 (+ 44608us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:00.886289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:00.897543 (+ 11254us) server_negotiation.cc:407] Connection header received
0811 01:45:00.897714 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:00.897719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:00.897936 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:00.898051 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:00.902311 (+ 4260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.903343 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.905877 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.906472 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.906605 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.911308 (+ 4703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:00.911317 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:00.911324 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:00.911368 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:00.915745 (+ 4377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.917431 (+ 1686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.917439 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.917445 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.917713 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:00.918311 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:00.918317 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:00.918320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:00.918691 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:00.918969 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:00.920073 (+ 1104us) server_negotiation.cc:299] Negotiation successful
0811 01:45:00.920359 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44467,"thread_start_us":47763,"threads_started":1}
I20260811 01:45:00.926604 29510 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:45:00.936894 29510 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:45:00.988021 31011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.963496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60920 (local address 127.21.214.65:34917)
0811 01:45:00.966109 (+ 2613us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:00.966135 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:00.967784 (+ 1649us) server_negotiation.cc:407] Connection header received
0811 01:45:00.968293 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:00.968308 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:00.968695 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:00.968968 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:00.970617 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.974098 (+ 3481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.984982 (+ 10884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.986398 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.986613 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.986960 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:00.987318 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 01:45:00.987560 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2349,"thread_start_us":2223,"threads_started":1}
I20260811 01:45:00.988535 31010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.963063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:34917 (local address 127.0.0.1:60920)
0811 01:45:00.966787 (+ 3724us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:00.966811 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:00.967935 (+ 1124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:00.969356 (+ 1421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:00.969363 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:00.969401 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:00.970370 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:00.970390 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.974377 (+ 3987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.974386 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:00.984770 (+ 10384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:00.984784 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.986749 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.986756 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:00.987045 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.987081 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 01:45:00.987199 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 01:45:00.987376 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3580,"mutex_wait_us":3,"thread_start_us":3417,"threads_started":1}
I20260811 01:45:00.990134 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.962277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:38921 (local address 127.0.0.1:52548)
0811 01:45:00.964373 (+ 2096us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:00.964427 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:00.964631 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:00.980147 (+ 15516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:00.980156 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:00.980244 (+ 88us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:00.981233 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:00.981260 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.983667 (+ 2407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.983686 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:00.986568 (+ 2882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:00.986587 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.989080 (+ 2493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.989088 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:00.989372 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.989397 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 01:45:00.989515 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 01:45:00.989649 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1855,"thread_start_us":1710,"threads_started":1}
I20260811 01:45:00.990262 31009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.962526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52548 (local address 127.21.214.66:38921)
0811 01:45:00.978920 (+ 16394us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:00.978934 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:00.978978 (+ 44us) server_negotiation.cc:407] Connection header received
0811 01:45:00.979197 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:00.979211 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:00.979761 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:00.981656 (+ 1895us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:00.981950 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.983445 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.986721 (+ 3276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.988755 (+ 2034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.988891 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.989228 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:00.989573 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 01:45:00.989827 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16114,"thread_start_us":15983,"threads_started":1}
I20260811 01:45:00.993450 30568 tablet_service.cc:1467] Processing CreateTablet for tablet b04159cc3b4b41028dbf89d3fd3bb4c9 (DEFAULT_TABLE table=test-table [id=f6d15305c430410da62bd03c7ca475fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:00.995222 31012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.966909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:38341 (local address 127.0.0.1:40326)
0811 01:45:00.982008 (+ 15099us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:00.982031 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:00.982180 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:00.982828 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:00.982834 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:00.982860 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:00.983524 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:00.983565 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.986746 (+ 3181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.986752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:00.991230 (+ 4478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:00.991246 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.993950 (+ 2704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.993957 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:00.994172 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.994221 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 01:45:00.994332 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 01:45:00.994502 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14949,"mutex_wait_us":25,"thread_start_us":14801,"threads_started":1}
I20260811 01:45:00.995298 31013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:00.967315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40326 (local address 127.21.214.67:38341)
0811 01:45:00.981819 (+ 14504us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:00.981830 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:00.982116 (+ 286us) server_negotiation.cc:407] Connection header received
0811 01:45:00.982365 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:00.982371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:00.982592 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:00.982860 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:00.984008 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.986481 (+ 2473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.991700 (+ 5219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:00.993581 (+ 1881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:00.993745 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:00.993985 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:00.994480 (+ 495us) server_negotiation.cc:299] Negotiation successful
0811 01:45:00.994691 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14230,"thread_start_us":14517,"threads_started":1}
I20260811 01:45:00.994715 30737 tablet_service.cc:1467] Processing CreateTablet for tablet b04159cc3b4b41028dbf89d3fd3bb4c9 (DEFAULT_TABLE table=test-table [id=f6d15305c430410da62bd03c7ca475fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:00.995729 30568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04159cc3b4b41028dbf89d3fd3bb4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:00.996627 30737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04159cc3b4b41028dbf89d3fd3bb4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:01.000552 30899 tablet_service.cc:1467] Processing CreateTablet for tablet b04159cc3b4b41028dbf89d3fd3bb4c9 (DEFAULT_TABLE table=test-table [id=f6d15305c430410da62bd03c7ca475fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:01.004501 30899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04159cc3b4b41028dbf89d3fd3bb4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:01.015882 31014 tablet_bootstrap.cc:492] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb: Bootstrap starting.
I20260811 01:45:01.018676 31015 tablet_bootstrap.cc:492] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b: Bootstrap starting.
I20260811 01:45:01.022255 31014 tablet_bootstrap.cc:654] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:01.024114 31014 log.cc:824] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb: Log is configured to *not* fsync() on all Append() calls
I20260811 01:45:01.024259 31015 tablet_bootstrap.cc:654] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:01.025067 31016 tablet_bootstrap.cc:492] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7: Bootstrap starting.
I20260811 01:45:01.027030 31015 log.cc:824] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b: Log is configured to *not* fsync() on all Append() calls
I20260811 01:45:01.029835 31014 tablet_bootstrap.cc:492] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb: No bootstrap required, opened a new log
I20260811 01:45:01.030261 31014 ts_tablet_manager.cc:1397] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.003s
I20260811 01:45:01.030877 31015 tablet_bootstrap.cc:492] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b: No bootstrap required, opened a new log
I20260811 01:45:01.031258 31015 ts_tablet_manager.cc:1397] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260811 01:45:01.031268 31016 tablet_bootstrap.cc:654] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:01.033272 31016 log.cc:824] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7: Log is configured to *not* fsync() on all Append() calls
I20260811 01:45:01.038100 31016 tablet_bootstrap.cc:492] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7: No bootstrap required, opened a new log
I20260811 01:45:01.038566 31016 ts_tablet_manager.cc:1397] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260811 01:45:01.042299 31015 raft_consensus.cc:348] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } } peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } }
I20260811 01:45:01.042912 31015 raft_consensus.cc:374] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:01.043108 31015 raft_consensus.cc:729] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56802ed9c3784965b20f771a4840de0b, State: Initialized, Role: FOLLOWER
I20260811 01:45:01.043973 31015 consensus_queue.cc:260] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } } peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } }
I20260811 01:45:01.045864 31014 raft_consensus.cc:348] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } } peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } }
I20260811 01:45:01.046475 31014 raft_consensus.cc:374] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:01.046633 31014 raft_consensus.cc:729] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26467dcaea844348b216edcec09ec9eb, State: Initialized, Role: FOLLOWER
I20260811 01:45:01.046860 31015 ts_tablet_manager.cc:1428] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b: Time spent starting tablet: real 0.015s user 0.010s sys 0.006s
I20260811 01:45:01.047307 30816 heartbeater.cc:503] Master 127.21.214.126:41815 was elected leader, sending a full tablet report...
I20260811 01:45:01.053876 31014 consensus_queue.cc:260] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } } peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } }
I20260811 01:45:01.055599 31023 raft_consensus.cc:482] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:45:01.056281 31023 raft_consensus.cc:504] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } } peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } }
I20260811 01:45:01.056650 31016 raft_consensus.cc:348] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } } peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } }
I20260811 01:45:01.057523 31016 raft_consensus.cc:374] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:01.057785 31016 raft_consensus.cc:729] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39a96547f8ec4bf08372b8ce43c159a7, State: Initialized, Role: FOLLOWER
I20260811 01:45:01.059073 31014 ts_tablet_manager.cc:1428] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb: Time spent starting tablet: real 0.029s user 0.022s sys 0.005s
I20260811 01:45:01.058931 31016 consensus_queue.cc:260] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } } peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } }
I20260811 01:45:01.064316 31023 leader_election.cc:288] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56802ed9c3784965b20f771a4840de0b (127.21.214.66:38921), 39a96547f8ec4bf08372b8ce43c159a7 (127.21.214.67:38341)
I20260811 01:45:01.065605 31022 raft_consensus.cc:482] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:45:01.066027 31022 raft_consensus.cc:504] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } } peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } }
I20260811 01:45:01.067705 31016 ts_tablet_manager.cc:1428] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7: Time spent starting tablet: real 0.029s user 0.019s sys 0.008s
I20260811 01:45:01.068706 30975 heartbeater.cc:503] Master 127.21.214.126:41815 was elected leader, sending a full tablet report...
I20260811 01:45:01.068403 31022 leader_election.cc:288] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26467dcaea844348b216edcec09ec9eb (127.21.214.65:34917), 39a96547f8ec4bf08372b8ce43c159a7 (127.21.214.67:38341)
I20260811 01:45:01.086829 31009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:01.064747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:46215 (local address 127.21.214.66:38921)
0811 01:45:01.065296 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:01.065305 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:01.070686 (+ 5381us) server_negotiation.cc:407] Connection header received
0811 01:45:01.070787 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:01.070794 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:01.070885 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:01.070985 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:01.076580 (+ 5595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.077972 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.082874 (+ 4902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.085205 (+ 2331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.085392 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:01.085497 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:01.086199 (+ 702us) server_negotiation.cc:299] Negotiation successful
0811 01:45:01.086356 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":248}
I20260811 01:45:01.087517 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:01.071883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:60135 (local address 127.21.214.67:38341)
0811 01:45:01.076601 (+ 4718us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:01.076605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:01.076628 (+ 23us) server_negotiation.cc:407] Connection header received
0811 01:45:01.076683 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:01.076687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:01.076736 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:01.076817 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:01.077730 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.079552 (+ 1822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.083496 (+ 3944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.085589 (+ 2093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.085802 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:01.085905 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:01.086706 (+ 801us) server_negotiation.cc:299] Negotiation successful
0811 01:45:01.086904 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4577,"thread_start_us":4352,"threads_started":1}
I20260811 01:45:01.088193 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:01.062914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:38921 (local address 127.21.214.65:46215)
0811 01:45:01.065485 (+ 2571us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:01.065521 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:01.065702 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:01.075598 (+ 9896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:01.075606 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:01.075658 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:01.076287 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:01.076306 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.078214 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.078220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:01.082683 (+ 4463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:01.082698 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.085751 (+ 3053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.085762 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:01.085978 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:01.086014 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 01:45:01.087362 (+ 1348us) client_negotiation.cc:240] Negotiation successful
0811 01:45:01.087515 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2400,"mutex_wait_us":80,"thread_start_us":5398,"threads_started":1}
I20260811 01:45:01.088670 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:01.071734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:38341 (local address 127.21.214.65:60135)
0811 01:45:01.074561 (+ 2827us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:01.074586 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:01.074801 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:01.076887 (+ 2086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:01.076892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:01.076931 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:01.077483 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:01.077496 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.079791 (+ 2295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.079796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:01.083219 (+ 3423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:01.083235 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.086198 (+ 2963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.086206 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:01.086485 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:01.086520 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 01:45:01.086599 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 01:45:01.088037 (+ 1438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2574,"mutex_wait_us":1277,"thread_start_us":3366,"threads_started":1}
I20260811 01:45:01.089501 31011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:01.073953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:58503 (local address 127.21.214.65:34917)
0811 01:45:01.074239 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:01.074248 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:01.076021 (+ 1773us) server_negotiation.cc:407] Connection header received
0811 01:45:01.077499 (+ 1478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:01.077519 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:01.077612 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:01.077918 (+ 306us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:01.079325 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.081666 (+ 2341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.086370 (+ 4704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.088028 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.088248 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:01.088322 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:01.089025 (+ 703us) server_negotiation.cc:299] Negotiation successful
0811 01:45:01.089186 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":76}
I20260811 01:45:01.090014 30761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04159cc3b4b41028dbf89d3fd3bb4c9" candidate_uuid: "26467dcaea844348b216edcec09ec9eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56802ed9c3784965b20f771a4840de0b" is_pre_election: true
I20260811 01:45:01.090629 31027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:01.068505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:34917 (local address 127.21.214.66:58503)
0811 01:45:01.075921 (+ 7416us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:01.075943 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:01.077317 (+ 1374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:01.078149 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:01.078158 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:01.078198 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:01.079098 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:01.079118 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.081954 (+ 2836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.081963 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:01.086124 (+ 4161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:01.086151 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.088604 (+ 2453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.088612 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:01.088832 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:01.088868 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 01:45:01.089679 (+ 811us) client_negotiation.cc:240] Negotiation successful
0811 01:45:01.089868 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2713,"mutex_wait_us":37,"thread_start_us":803,"threads_started":1}
I20260811 01:45:01.090648 31013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:01.069939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:38259 (local address 127.21.214.67:38341)
0811 01:45:01.070130 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:01.070146 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:01.080444 (+ 10298us) server_negotiation.cc:407] Connection header received
0811 01:45:01.080636 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:01.080644 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:01.080714 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:01.080830 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:01.082337 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.083776 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.087363 (+ 3587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.089003 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.089191 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:01.089278 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:01.090145 (+ 867us) server_negotiation.cc:299] Negotiation successful
0811 01:45:01.090327 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 01:45:01.090866 30761 raft_consensus.cc:2454] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26467dcaea844348b216edcec09ec9eb in term 0.
I20260811 01:45:01.091101 30921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04159cc3b4b41028dbf89d3fd3bb4c9" candidate_uuid: "26467dcaea844348b216edcec09ec9eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39a96547f8ec4bf08372b8ce43c159a7" is_pre_election: true
I20260811 01:45:01.091840 30921 raft_consensus.cc:2454] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26467dcaea844348b216edcec09ec9eb in term 0.
I20260811 01:45:01.092206 30517 leader_election.cc:302] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26467dcaea844348b216edcec09ec9eb, 56802ed9c3784965b20f771a4840de0b; no voters:
I20260811 01:45:01.093448 31028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:01.069751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:38341 (local address 127.21.214.66:38259)
0811 01:45:01.080332 (+ 10581us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:01.080347 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:01.080467 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:01.081049 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:01.081053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:01.081080 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:01.082050 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:01.082067 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.084056 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.084074 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:01.087136 (+ 3062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:01.087151 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.089551 (+ 2400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.089560 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:01.089877 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:01.089917 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 01:45:01.090884 (+ 967us) client_negotiation.cc:240] Negotiation successful
0811 01:45:01.091163 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10465,"mutex_wait_us":1540,"thread_start_us":8708,"threads_started":1}
I20260811 01:45:01.093412 31023 raft_consensus.cc:2781] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:45:01.093690 31023 raft_consensus.cc:482] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:45:01.093919 31023 raft_consensus.cc:3037] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:01.093724 30590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04159cc3b4b41028dbf89d3fd3bb4c9" candidate_uuid: "56802ed9c3784965b20f771a4840de0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26467dcaea844348b216edcec09ec9eb" is_pre_election: true
I20260811 01:45:01.094337 30921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04159cc3b4b41028dbf89d3fd3bb4c9" candidate_uuid: "56802ed9c3784965b20f771a4840de0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39a96547f8ec4bf08372b8ce43c159a7" is_pre_election: true
I20260811 01:45:01.094616 30921 raft_consensus.cc:2454] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56802ed9c3784965b20f771a4840de0b in term 0.
I20260811 01:45:01.095167 30683 leader_election.cc:302] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39a96547f8ec4bf08372b8ce43c159a7, 56802ed9c3784965b20f771a4840de0b; no voters:
I20260811 01:45:01.095655 31022 raft_consensus.cc:2781] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:45:01.095863 31022 raft_consensus.cc:482] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:45:01.096024 31022 raft_consensus.cc:3037] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:01.097905 31023 raft_consensus.cc:504] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } } peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } }
I20260811 01:45:01.098767 30590 raft_consensus.cc:2379] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56802ed9c3784965b20f771a4840de0b in current term 1: Already voted for candidate 26467dcaea844348b216edcec09ec9eb in this term.
I20260811 01:45:01.099660 31023 leader_election.cc:288] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [CANDIDATE]: Term 1 election: Requested vote from peers 56802ed9c3784965b20f771a4840de0b (127.21.214.66:38921), 39a96547f8ec4bf08372b8ce43c159a7 (127.21.214.67:38341)
I20260811 01:45:01.100095 30761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04159cc3b4b41028dbf89d3fd3bb4c9" candidate_uuid: "26467dcaea844348b216edcec09ec9eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56802ed9c3784965b20f771a4840de0b"
I20260811 01:45:01.100097 30921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04159cc3b4b41028dbf89d3fd3bb4c9" candidate_uuid: "26467dcaea844348b216edcec09ec9eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39a96547f8ec4bf08372b8ce43c159a7"
I20260811 01:45:01.100324 30921 raft_consensus.cc:3037] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:01.100986 31022 raft_consensus.cc:504] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } } peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } }
I20260811 01:45:01.101645 30761 raft_consensus.cc:2379] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 26467dcaea844348b216edcec09ec9eb in current term 1: Already voted for candidate 56802ed9c3784965b20f771a4840de0b in this term.
I20260811 01:45:01.102054 31022 leader_election.cc:288] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [CANDIDATE]: Term 1 election: Requested vote from peers 26467dcaea844348b216edcec09ec9eb (127.21.214.65:34917), 39a96547f8ec4bf08372b8ce43c159a7 (127.21.214.67:38341)
I20260811 01:45:01.103130 30920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04159cc3b4b41028dbf89d3fd3bb4c9" candidate_uuid: "56802ed9c3784965b20f771a4840de0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39a96547f8ec4bf08372b8ce43c159a7"
I20260811 01:45:01.103523 30590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04159cc3b4b41028dbf89d3fd3bb4c9" candidate_uuid: "56802ed9c3784965b20f771a4840de0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26467dcaea844348b216edcec09ec9eb"
I20260811 01:45:01.103823 30590 raft_consensus.cc:2379] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 56802ed9c3784965b20f771a4840de0b in current term 1: Already voted for candidate 26467dcaea844348b216edcec09ec9eb in this term.
I20260811 01:45:01.104406 30683 leader_election.cc:302] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56802ed9c3784965b20f771a4840de0b; no voters: 26467dcaea844348b216edcec09ec9eb, 39a96547f8ec4bf08372b8ce43c159a7
I20260811 01:45:01.104792 30921 raft_consensus.cc:2454] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26467dcaea844348b216edcec09ec9eb in term 1.
I20260811 01:45:01.104935 31022 raft_consensus.cc:2726] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260811 01:45:01.105276 30515 leader_election.cc:302] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26467dcaea844348b216edcec09ec9eb, 39a96547f8ec4bf08372b8ce43c159a7; no voters: 56802ed9c3784965b20f771a4840de0b
I20260811 01:45:01.105747 31023 raft_consensus.cc:2781] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:45:01.105931 31023 raft_consensus.cc:686] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [term 1 LEADER]: Becoming Leader. State: Replica: 26467dcaea844348b216edcec09ec9eb, State: Running, Role: LEADER
I20260811 01:45:01.106554 31023 consensus_queue.cc:237] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } } peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } }
I20260811 01:45:01.112917 29512 catalog_manager.cc:5495] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 26467dcaea844348b216edcec09ec9eb (127.21.214.65). New cstate: current_term: 1 leader_uuid: "26467dcaea844348b216edcec09ec9eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } health_report { overall_health: UNKNOWN } } }
W20260811 01:45:01.147099 30818 tablet.cc:2369] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:45:01.174811 31011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:01.168604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60928 (local address 127.21.214.65:34917)
0811 01:45:01.168819 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:01.168825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:01.168851 (+ 26us) server_negotiation.cc:407] Connection header received
0811 01:45:01.169026 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:01.169030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:01.169111 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:01.169320 (+ 209us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:45:01.169685 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.170584 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.172286 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.172919 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.173021 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:01.173589 (+ 568us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:45:01.174128 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:45:01.174246 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:01.174379 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 01:45:01.174450 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 01:45:01.185001 30921 raft_consensus.cc:1261] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7 [term 1 FOLLOWER]: Refusing update from remote peer 26467dcaea844348b216edcec09ec9eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:45:01.185214 30761 raft_consensus.cc:1261] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [term 1 FOLLOWER]: Refusing update from remote peer 26467dcaea844348b216edcec09ec9eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:45:01.186216 31023 consensus_queue.cc:1035] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:45:01.186594 31024 consensus_queue.cc:1035] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 01:45:01.193413 30648 tablet.cc:2369] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:45:01.239962 29512 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:45:01.240389 29512 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:45:01.248507 29512 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 662c631857f64f2c96f1a494f2e6c01d: Sending DeleteTablet for 3 replicas of tablet b04159cc3b4b41028dbf89d3fd3bb4c9
I20260811 01:45:01.250641 30568 tablet_service.cc:1514] Processing DeleteTablet for tablet b04159cc3b4b41028dbf89d3fd3bb4c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:01 UTC) from {username='oryx', principal='oryx/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:60920
I20260811 01:45:01.252120 30737 tablet_service.cc:1514] Processing DeleteTablet for tablet b04159cc3b4b41028dbf89d3fd3bb4c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:01 UTC) from {username='oryx', principal='oryx/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:52548
I20260811 01:45:01.253204 30899 tablet_service.cc:1514] Processing DeleteTablet for tablet b04159cc3b4b41028dbf89d3fd3bb4c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:01 UTC) from {username='oryx', principal='oryx/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:40326
I20260811 01:45:01.258868 31044 tablet_replica.cc:330] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb: stopping tablet replica
I20260811 01:45:01.259567 31044 raft_consensus.cc:2229] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:45:01.260839 31044 raft_consensus.cc:2258] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:01.265084 31044 ts_tablet_manager.cc:1905] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:01.270612 31048 tablet_replica.cc:330] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7: stopping tablet replica
I20260811 01:45:01.271425 31048 raft_consensus.cc:2229] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:45:01.272311 31048 raft_consensus.cc:2258] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:01.274089 31047 tablet_replica.cc:330] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b: stopping tablet replica
I20260811 01:45:01.276798 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 29473
I20260811 01:45:01.277318 31048 ts_tablet_manager.cc:1905] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:01.278501 29546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:01.274829 31047 raft_consensus.cc:2229] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:45:01.284061 31047 raft_consensus.cc:2258] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:01.284178 31044 ts_tablet_manager.cc:1918] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:45:01.284543 31044 log.cc:1196] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/b04159cc3b4b41028dbf89d3fd3bb4c9
I20260811 01:45:01.285444 31044 ts_tablet_manager.cc:1939] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 26467dcaea844348b216edcec09ec9eb: Deleting consensus metadata
I20260811 01:45:01.287081 31047 ts_tablet_manager.cc:1905] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:01.299069 31047 ts_tablet_manager.cc:1918] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:45:01.299324 31047 log.cc:1196] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/b04159cc3b4b41028dbf89d3fd3bb4c9
I20260811 01:45:01.300498 31047 ts_tablet_manager.cc:1939] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 56802ed9c3784965b20f771a4840de0b: Deleting consensus metadata
I20260811 01:45:01.303531 31048 ts_tablet_manager.cc:1918] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:45:01.303866 31048 log.cc:1196] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/b04159cc3b4b41028dbf89d3fd3bb4c9
I20260811 01:45:01.304862 31048 ts_tablet_manager.cc:1939] T b04159cc3b4b41028dbf89d3fd3bb4c9 P 39a96547f8ec4bf08372b8ce43c159a7: Deleting consensus metadata
I20260811 01:45:01.310814 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:01.256552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45720 (local address 127.21.214.125:41479)
0811 01:45:01.256791 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:01.256801 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:01.275025 (+ 18224us) server_negotiation.cc:407] Connection header received
0811 01:45:01.275121 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:01.275126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:01.275219 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:01.275328 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:01.284843 (+ 9515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.286355 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.289226 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.290175 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.290401 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:01.301041 (+ 10640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:01.301052 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:01.301060 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:01.301111 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:01.307060 (+ 5949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:01.308094 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:01.308100 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:01.308104 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:01.308269 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:01.308792 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:01.308798 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:01.308803 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:01.309275 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:01.309541 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:01.310084 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 01:45:01.310322 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 01:45:01.330191 30981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:01.274844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40170 (local address 127.21.214.124:39589)
0811 01:45:01.275042 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:01.275048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:01.311203 (+ 36155us) server_negotiation.cc:407] Connection header received
0811 01:45:01.311298 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:01.311304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:01.311400 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:01.311515 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:01.314593 (+ 3078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.315945 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.317901 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.318476 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.318669 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:01.322884 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:01.322890 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:01.322894 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:01.323053 (+ 159us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:01.326418 (+ 3365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:01.327361 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:01.327369 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:01.327378 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:01.327552 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:01.328050 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:01.328055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:01.328057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:01.328462 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:01.328854 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:01.329236 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 01:45:01.329569 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 01:45:01.522540 29493 catalog_manager.cc:4841] TS 26467dcaea844348b216edcec09ec9eb (127.21.214.65:34917): tablet b04159cc3b4b41028dbf89d3fd3bb4c9 (table test-table [id=f6d15305c430410da62bd03c7ca475fd]) successfully deleted
I20260811 01:45:01.523043 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 29473
W20260811 01:45:01.546702 30847 connection.cc:535] server connection from 127.0.0.1:40326 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 01:45:01.590363 30981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:01.565888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40186 (local address 127.21.214.124:39589)
0811 01:45:01.566423 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:01.566434 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:01.569424 (+ 2990us) server_negotiation.cc:407] Connection header received
0811 01:45:01.570025 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:01.570035 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:01.570149 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:01.570244 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:01.574314 (+ 4070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.575506 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.577811 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.578506 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.578652 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:01.582956 (+ 4304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:01.582964 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:01.582970 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:01.583020 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:01.586210 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:01.587163 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:01.587170 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:01.587174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:01.587353 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:01.587908 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:01.587915 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:01.587919 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:01.588351 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:01.588647 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:01.589611 (+ 964us) server_negotiation.cc:299] Negotiation successful
0811 01:45:01.589868 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330}
I20260811 01:45:01.592135 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:01.566128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45728 (local address 127.21.214.125:41479)
0811 01:45:01.566649 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:01.566658 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:01.574986 (+ 8328us) server_negotiation.cc:407] Connection header received
0811 01:45:01.575292 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:01.575299 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:01.575403 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:01.575479 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:01.577671 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.578795 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.580466 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:01.580904 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:01.581056 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:01.584217 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:01.584224 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:01.584229 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:01.584273 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:01.587863 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:01.588485 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:01.588490 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:01.588494 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:01.588693 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:01.589110 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:01.589115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:01.589118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:01.589534 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:01.590267 (+ 733us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:01.591418 (+ 1151us) server_negotiation.cc:299] Negotiation successful
0811 01:45:01.591748 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330}
I20260811 01:45:02.216552 31058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:02.215729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.66:45449)
0811 01:45:02.216261 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:02.216388 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":277,"threads_started":1}
W20260811 01:45:02.217134 30816 heartbeater.cc:644] Failed to heartbeat to 127.21.214.126:41815 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
I20260811 01:45:02.226982 31059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:02.225757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.65:48789)
0811 01:45:02.226317 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:02.226478 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"threads_started":1}
W20260811 01:45:02.227638 30645 heartbeater.cc:644] Failed to heartbeat to 127.21.214.126:41815 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
I20260811 01:45:02.228097 31060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:02.227309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.67:50809)
0811 01:45:02.227828 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:02.227936 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":274,"threads_started":1}
W20260811 01:45:02.228531 30975 heartbeater.cc:644] Failed to heartbeat to 127.21.214.126:41815 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
I20260811 01:45:02.749382 31063 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 662c631857f64f2c96f1a494f2e6c01d)
I20260811 01:45:02.749375 31062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 662c631857f64f2c96f1a494f2e6c01d)
I20260811 01:45:02.749701 31062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } }
I20260811 01:45:02.749694 31063 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } }
I20260811 01:45:02.751190 31062 leader_election.cc:288] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 662c631857f64f2c96f1a494f2e6c01d (127.21.214.126:41815), bbd0420e2f7342008f02ce52a2778fbf (127.21.214.125:41479)
I20260811 01:45:02.752666 31065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:02.751339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:35516)
0811 01:45:02.752003 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:02.752294 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":515,"threads_started":1}
I20260811 01:45:02.753890 31063 leader_election.cc:288] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 662c631857f64f2c96f1a494f2e6c01d (127.21.214.126:41815), 22b5c1f3bba6482d8e5d02f5b8b2497d (127.21.214.124:39589)
W20260811 01:45:02.755322 30419 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
I20260811 01:45:02.756422 31064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:02.754380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:35528)
0811 01:45:02.755928 (+ 1548us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:02.756152 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1401,"thread_start_us":897,"threads_started":1}
W20260811 01:45:02.756731 30184 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
I20260811 01:45:02.757478 31066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:02.756782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:35544)
0811 01:45:02.757023 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:02.757222 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260811 01:45:02.757833 30419 leader_election.cc:334] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 662c631857f64f2c96f1a494f2e6c01d (127.21.214.126:41815): Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
I20260811 01:45:02.759534 31064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:02.759168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:35554)
0811 01:45:02.759338 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:02.759403 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 01:45:02.759773 30184 leader_election.cc:334] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 662c631857f64f2c96f1a494f2e6c01d (127.21.214.126:41815): Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
I20260811 01:45:02.763209 31065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:02.751428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:41479 (local address 127.0.0.1:51476)
0811 01:45:02.753108 (+ 1680us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:02.753133 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:02.753353 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:02.754740 (+ 1387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:02.754752 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:02.754797 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:02.755153 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:02.755172 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:02.758707 (+ 3535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:02.758713 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:02.760792 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:02.760808 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:02.762427 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:02.762431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:02.762568 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:02.762594 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 01:45:02.762730 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 01:45:02.762880 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1533,"thread_start_us":2200,"threads_started":1}
I20260811 01:45:02.763258 31067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:02.751632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51476 (local address 127.21.214.125:41479)
0811 01:45:02.754308 (+ 2676us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:02.754315 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:02.754335 (+ 20us) server_negotiation.cc:407] Connection header received
0811 01:45:02.754407 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:02.754412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:02.754534 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:02.756248 (+ 1714us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:02.756622 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:02.758531 (+ 1909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:02.760968 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:02.762245 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:02.762404 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:02.762484 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:02.762778 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 01:45:02.762892 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":2519,"thread_start_us":2317,"threads_started":1}
I20260811 01:45:02.763695 31068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:02.752184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44182 (local address 127.21.214.124:39589)
0811 01:45:02.755114 (+ 2930us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:02.755121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:02.755848 (+ 727us) server_negotiation.cc:407] Connection header received
0811 01:45:02.755921 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:02.755929 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:02.756013 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:02.756135 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:02.757358 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:02.758792 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:02.761254 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:02.762549 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:02.762673 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:02.762764 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:02.763166 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 01:45:02.763328 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":2787,"thread_start_us":2645,"threads_started":1}
I20260811 01:45:02.763970 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:02.751363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:39589 (local address 127.0.0.1:44182)
0811 01:45:02.755595 (+ 4232us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:02.755619 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:02.755772 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:02.756425 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:02.756430 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:02.756457 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:02.757064 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:02.757081 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:02.758937 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:02.758944 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:02.761073 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:02.761086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:02.762728 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:02.762735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:02.762914 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:02.762950 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 01:45:02.763061 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 01:45:02.763190 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4074,"mutex_wait_us":66,"thread_start_us":4246,"threads_started":1}
I20260811 01:45:02.764147 30211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bbd0420e2f7342008f02ce52a2778fbf" is_pre_election: true
I20260811 01:45:02.764533 30211 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22b5c1f3bba6482d8e5d02f5b8b2497d in term 1.
I20260811 01:45:02.764624 30445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bbd0420e2f7342008f02ce52a2778fbf" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" is_pre_election: true
I20260811 01:45:02.764946 30445 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbd0420e2f7342008f02ce52a2778fbf in term 1.
I20260811 01:45:02.765203 30415 leader_election.cc:302] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22b5c1f3bba6482d8e5d02f5b8b2497d, bbd0420e2f7342008f02ce52a2778fbf; no voters: 662c631857f64f2c96f1a494f2e6c01d
I20260811 01:45:02.765545 30182 leader_election.cc:302] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22b5c1f3bba6482d8e5d02f5b8b2497d, bbd0420e2f7342008f02ce52a2778fbf; no voters: 662c631857f64f2c96f1a494f2e6c01d
I20260811 01:45:02.765620 31062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 01:45:02.765746 31062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 662c631857f64f2c96f1a494f2e6c01d)
I20260811 01:45:02.765854 31062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:45:02.765897 31063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 01:45:02.766016 31063 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 1 FOLLOWER]: Starting leader election (detected failure of leader 662c631857f64f2c96f1a494f2e6c01d)
I20260811 01:45:02.766126 31063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:45:02.769093 31062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } }
I20260811 01:45:02.769116 31063 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } }
I20260811 01:45:02.769898 31062 leader_election.cc:288] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [CANDIDATE]: Term 2 election: Requested vote from peers 662c631857f64f2c96f1a494f2e6c01d (127.21.214.126:41815), bbd0420e2f7342008f02ce52a2778fbf (127.21.214.125:41479)
I20260811 01:45:02.769928 31063 leader_election.cc:288] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [CANDIDATE]: Term 2 election: Requested vote from peers 662c631857f64f2c96f1a494f2e6c01d (127.21.214.126:41815), 22b5c1f3bba6482d8e5d02f5b8b2497d (127.21.214.124:39589)
I20260811 01:45:02.770478 30445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bbd0420e2f7342008f02ce52a2778fbf" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d"
I20260811 01:45:02.770589 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:02.770134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:35576)
0811 01:45:02.770332 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:02.770428 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 01:45:02.770684 31065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:02.769998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:35570)
0811 01:45:02.770426 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:02.770507 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329}
I20260811 01:45:02.770819 30445 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bbd0420e2f7342008f02ce52a2778fbf in current term 2: Already voted for candidate 22b5c1f3bba6482d8e5d02f5b8b2497d in this term.
I20260811 01:45:02.770848 30211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bbd0420e2f7342008f02ce52a2778fbf"
I20260811 01:45:02.771121 30211 raft_consensus.cc:2379] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 22b5c1f3bba6482d8e5d02f5b8b2497d in current term 2: Already voted for candidate bbd0420e2f7342008f02ce52a2778fbf in this term.
I20260811 01:45:02.771808 31065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:02.771428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:35588)
0811 01:45:02.771572 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:02.771693 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 01:45:02.772017 30419 leader_election.cc:334] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 662c631857f64f2c96f1a494f2e6c01d (127.21.214.126:41815): Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
I20260811 01:45:02.772157 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:02.771725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:35600)
0811 01:45:02.771887 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:02.771998 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 01:45:02.772187 30419 leader_election.cc:302] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 22b5c1f3bba6482d8e5d02f5b8b2497d; no voters: 662c631857f64f2c96f1a494f2e6c01d, bbd0420e2f7342008f02ce52a2778fbf
I20260811 01:45:02.772505 31062 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260811 01:45:02.773073 30184 leader_election.cc:334] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 662c631857f64f2c96f1a494f2e6c01d (127.21.214.126:41815): Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
I20260811 01:45:02.773406 30184 leader_election.cc:302] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bbd0420e2f7342008f02ce52a2778fbf; no voters: 22b5c1f3bba6482d8e5d02f5b8b2497d, 662c631857f64f2c96f1a494f2e6c01d
I20260811 01:45:02.773703 31063 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 01:45:03.218652 31072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:03.217897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.66:45081)
0811 01:45:03.218369 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:03.218470 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":252,"threads_started":1}
I20260811 01:45:03.229291 31073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:03.228355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.65:53303)
0811 01:45:03.228877 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:03.229000 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":313,"threads_started":1}
I20260811 01:45:03.230022 31074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:03.229250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.67:39433)
0811 01:45:03.229766 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:03.229867 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":361,"threads_started":1}
I20260811 01:45:04.220625 31075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.219681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.66:54535)
0811 01:45:04.220291 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.220413 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":347,"threads_started":1}
I20260811 01:45:04.231338 31076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.230260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.65:54893)
0811 01:45:04.230937 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.231057 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":412,"threads_started":1}
I20260811 01:45:04.231671 31077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.230937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.67:33353)
0811 01:45:04.231351 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.231464 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":230,"threads_started":1}
I20260811 01:45:04.561113 31078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:45:04.561455 31078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } }
I20260811 01:45:04.562182 31078 leader_election.cc:288] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 662c631857f64f2c96f1a494f2e6c01d (127.21.214.126:41815), bbd0420e2f7342008f02ce52a2778fbf (127.21.214.125:41479)
I20260811 01:45:04.562960 30211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bbd0420e2f7342008f02ce52a2778fbf" is_pre_election: true
I20260811 01:45:04.563288 30211 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22b5c1f3bba6482d8e5d02f5b8b2497d in term 2.
I20260811 01:45:04.563442 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.562367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:35872)
0811 01:45:04.562969 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.563119 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":261,"threads_started":1}
I20260811 01:45:04.564007 30415 leader_election.cc:302] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22b5c1f3bba6482d8e5d02f5b8b2497d, bbd0420e2f7342008f02ce52a2778fbf; no voters:
I20260811 01:45:04.564383 31078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 01:45:04.564543 31078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:45:04.564684 31078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:45:04.565227 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.564819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:35886)
0811 01:45:04.564962 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.565042 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 01:45:04.565507 30419 leader_election.cc:334] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 662c631857f64f2c96f1a494f2e6c01d (127.21.214.126:41815): Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
I20260811 01:45:04.567145 31078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } }
I20260811 01:45:04.567739 31078 leader_election.cc:288] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [CANDIDATE]: Term 3 election: Requested vote from peers 662c631857f64f2c96f1a494f2e6c01d (127.21.214.126:41815), bbd0420e2f7342008f02ce52a2778fbf (127.21.214.125:41479)
I20260811 01:45:04.568353 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.567821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:35894)
0811 01:45:04.568030 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.568172 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 01:45:04.568325 30211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bbd0420e2f7342008f02ce52a2778fbf"
I20260811 01:45:04.568614 30211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:45:04.569486 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.569041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:35906)
0811 01:45:04.569223 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.569314 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 01:45:04.569686 30419 leader_election.cc:334] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 662c631857f64f2c96f1a494f2e6c01d (127.21.214.126:41815): Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
I20260811 01:45:04.571552 30211 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22b5c1f3bba6482d8e5d02f5b8b2497d in term 3.
I20260811 01:45:04.572017 30415 leader_election.cc:302] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22b5c1f3bba6482d8e5d02f5b8b2497d, bbd0420e2f7342008f02ce52a2778fbf; no voters: 662c631857f64f2c96f1a494f2e6c01d
I20260811 01:45:04.572368 31078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 3 FOLLOWER]: Leader election won for term 3
I20260811 01:45:04.573251 31078 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [term 3 LEADER]: Becoming Leader. State: Replica: 22b5c1f3bba6482d8e5d02f5b8b2497d, State: Running, Role: LEADER
I20260811 01:45:04.573781 31078 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } }
I20260811 01:45:04.576083 31082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22b5c1f3bba6482d8e5d02f5b8b2497d. Latest consensus state: current_term: 3 leader_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } } }
I20260811 01:45:04.576498 31082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [sys.catalog]: This master's current role is: LEADER
I20260811 01:45:04.577287 31084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:45:04.581800 31084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:45:04.582731 31084 catalog_manager.cc:1259] Loaded cluster ID: 61662ac4f271493a9ff65577e0e6c353
I20260811 01:45:04.582839 31084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:45:04.584229 31084 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:45:04.585232 31084 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: Loaded TSK: 0
I20260811 01:45:04.586241 31084 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:45:04.636992 30432 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40186:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:45:04.645067 30432 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:45:04.650070 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.649575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:35920)
0811 01:45:04.649788 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.649907 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 01:45:04.650182 30211 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [term 3 FOLLOWER]: Refusing update from remote peer 22b5c1f3bba6482d8e5d02f5b8b2497d: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 01:45:04.651039 31078 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 01:45:04.651197 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.650775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:35924)
0811 01:45:04.650992 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.651069 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 01:45:04.651444 30419 consensus_peers.cc:487] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d -> Peer 662c631857f64f2c96f1a494f2e6c01d (127.21.214.126:41815): Couldn't send request to peer 662c631857f64f2c96f1a494f2e6c01d. Status: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 01:45:04.654510 31086 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22b5c1f3bba6482d8e5d02f5b8b2497d. Latest consensus state: current_term: 3 leader_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } } }
I20260811 01:45:04.655054 31086 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:45:04.656113 31078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } } }
I20260811 01:45:04.656478 31078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d [sys.catalog]: This master's current role is: LEADER
I20260811 01:45:04.657063 31086 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "662c631857f64f2c96f1a494f2e6c01d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 41815 } } peers { permanent_uuid: "bbd0420e2f7342008f02ce52a2778fbf" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 41479 } attrs { promote: false } } peers { permanent_uuid: "22b5c1f3bba6482d8e5d02f5b8b2497d" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 39589 } attrs { promote: false } } }
I20260811 01:45:04.657521 31086 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd0420e2f7342008f02ce52a2778fbf [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:45:04.685253 31097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.667300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49748 (local address 127.21.214.67:38341)
0811 01:45:04.672205 (+ 4905us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:04.672215 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:04.672244 (+ 29us) server_negotiation.cc:407] Connection header received
0811 01:45:04.672359 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:04.672367 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:04.672484 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:04.672631 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:04.676069 (+ 3438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:04.677640 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:04.682906 (+ 5266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:04.684159 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:04.684321 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:04.684494 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:04.684769 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 01:45:04.684913 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4704,"thread_start_us":4555,"threads_started":1}
I20260811 01:45:04.685247 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.666296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:38341 (local address 127.0.0.1:49748)
0811 01:45:04.666547 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.666571 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:04.666720 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:04.672679 (+ 5959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:04.672689 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:04.672735 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:04.675775 (+ 3040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:04.675798 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:04.678284 (+ 2486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:04.678294 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:04.681313 (+ 3019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:04.681336 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:04.684368 (+ 3032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:04.684374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:04.684585 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:04.684612 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 01:45:04.684730 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 01:45:04.684886 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123}
I20260811 01:45:04.686661 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.666921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:38921 (local address 127.0.0.1:43262)
0811 01:45:04.678715 (+ 11794us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.678738 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:04.678903 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:04.679320 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:04.679325 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:04.679359 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:04.679958 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:04.679978 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:04.681325 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:04.681335 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:04.684295 (+ 2960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:04.684315 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:04.685828 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:04.685838 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:04.686028 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:04.686062 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 01:45:04.686155 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 01:45:04.686284 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11645,"thread_start_us":11487,"threads_started":1}
I20260811 01:45:04.686544 30899 tablet_service.cc:1467] Processing CreateTablet for tablet 1266b3d9244e4434b51809cd5ce79e58 (DEFAULT_TABLE table=test-table [id=b942f9df28c144c08b25657d576a8642]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:04.686939 31096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.667060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43262 (local address 127.21.214.66:38921)
0811 01:45:04.675781 (+ 8721us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:04.675790 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:04.679010 (+ 3220us) server_negotiation.cc:407] Connection header received
0811 01:45:04.679108 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:04.679112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:04.679178 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:04.679303 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:04.680157 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:04.681130 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:04.684476 (+ 3346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:04.685545 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:04.685715 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:04.685830 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:04.686352 (+ 522us) server_negotiation.cc:299] Negotiation successful
0811 01:45:04.686533 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8514,"thread_start_us":8915,"threads_started":1}
I20260811 01:45:04.687146 30899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1266b3d9244e4434b51809cd5ce79e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:04.691123 31098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.667469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:34917 (local address 127.0.0.1:52936)
0811 01:45:04.682579 (+ 15110us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.682600 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:04.682732 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:04.683223 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:04.683228 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:04.683254 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:04.683762 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:04.683787 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:04.686375 (+ 2588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:04.686380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:04.688776 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:04.688790 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:04.690225 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:04.690232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:04.690407 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:04.690434 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 01:45:04.690558 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 01:45:04.690702 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14919,"thread_start_us":14689,"threads_started":1}
I20260811 01:45:04.691123 31099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.667664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52936 (local address 127.21.214.65:34917)
0811 01:45:04.682865 (+ 15201us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:04.682873 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:04.682896 (+ 23us) server_negotiation.cc:407] Connection header received
0811 01:45:04.682975 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:04.682981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:04.683062 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:04.683177 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:04.684038 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:04.686185 (+ 2147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:04.688980 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:04.689966 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:04.690131 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:04.690269 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:04.690590 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 01:45:04.690775 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15034,"thread_start_us":14862,"threads_started":1}
I20260811 01:45:04.691695 30737 tablet_service.cc:1467] Processing CreateTablet for tablet 1266b3d9244e4434b51809cd5ce79e58 (DEFAULT_TABLE table=test-table [id=b942f9df28c144c08b25657d576a8642]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:04.692195 30737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1266b3d9244e4434b51809cd5ce79e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:04.692734 30568 tablet_service.cc:1467] Processing CreateTablet for tablet 1266b3d9244e4434b51809cd5ce79e58 (DEFAULT_TABLE table=test-table [id=b942f9df28c144c08b25657d576a8642]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:04.693261 30568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1266b3d9244e4434b51809cd5ce79e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:04.697396 31100 tablet_bootstrap.cc:492] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7: Bootstrap starting.
I20260811 01:45:04.699259 31100 tablet_bootstrap.cc:654] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:04.702119 31100 tablet_bootstrap.cc:492] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7: No bootstrap required, opened a new log
I20260811 01:45:04.702296 31100 ts_tablet_manager.cc:1397] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260811 01:45:04.703073 31100 raft_consensus.cc:348] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } } peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } }
I20260811 01:45:04.703299 31100 raft_consensus.cc:374] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:04.703409 31100 raft_consensus.cc:729] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39a96547f8ec4bf08372b8ce43c159a7, State: Initialized, Role: FOLLOWER
I20260811 01:45:04.703708 31100 consensus_queue.cc:260] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } } peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } }
I20260811 01:45:04.704830 31100 ts_tablet_manager.cc:1428] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 01:45:04.705462 31077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.704948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.67:56819)
0811 01:45:04.705120 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.705287 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 01:45:04.705681 31102 tablet_bootstrap.cc:492] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb: Bootstrap starting.
I20260811 01:45:04.706599 31077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.706139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.67:38217)
0811 01:45:04.706349 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.706434 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260811 01:45:04.707983 30974 heartbeater.cc:503] Master 127.21.214.124:39589 was elected leader, sending a full tablet report...
I20260811 01:45:04.708678 31102 tablet_bootstrap.cc:654] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:04.711304 31102 tablet_bootstrap.cc:492] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb: No bootstrap required, opened a new log
I20260811 01:45:04.711506 31102 ts_tablet_manager.cc:1397] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 01:45:04.712442 31101 tablet_bootstrap.cc:492] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b: Bootstrap starting.
I20260811 01:45:04.712520 31102 raft_consensus.cc:348] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } } peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } }
I20260811 01:45:04.712776 31102 raft_consensus.cc:374] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:04.712882 31102 raft_consensus.cc:729] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26467dcaea844348b216edcec09ec9eb, State: Initialized, Role: FOLLOWER
I20260811 01:45:04.713194 31102 consensus_queue.cc:260] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } } peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } }
I20260811 01:45:04.714294 31102 ts_tablet_manager.cc:1428] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 01:45:04.714788 31101 tablet_bootstrap.cc:654] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:04.715421 31076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.714886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.65:45011)
0811 01:45:04.715130 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.715242 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260811 01:45:04.716434 30644 heartbeater.cc:503] Master 127.21.214.124:39589 was elected leader, sending a full tablet report...
I20260811 01:45:04.716826 31076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.716104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.65:36575)
0811 01:45:04.716422 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.716529 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260811 01:45:04.718513 31101 tablet_bootstrap.cc:492] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b: No bootstrap required, opened a new log
I20260811 01:45:04.718673 31101 ts_tablet_manager.cc:1397] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20260811 01:45:04.719372 31101 raft_consensus.cc:348] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } } peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } }
I20260811 01:45:04.719550 31101 raft_consensus.cc:374] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:04.719674 31101 raft_consensus.cc:729] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56802ed9c3784965b20f771a4840de0b, State: Initialized, Role: FOLLOWER
I20260811 01:45:04.719947 31101 consensus_queue.cc:260] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } } peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } }
I20260811 01:45:04.720717 31101 ts_tablet_manager.cc:1428] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260811 01:45:04.721843 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.721116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.66:41861)
0811 01:45:04.721562 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.721655 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":243,"threads_started":1}
I20260811 01:45:04.722956 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.722535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.66:36277)
0811 01:45:04.722705 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.722798 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 01:45:04.723353 30815 heartbeater.cc:503] Master 127.21.214.124:39589 was elected leader, sending a full tablet report...
W20260811 01:45:04.813939 30977 tablet.cc:2369] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:45:04.831902 31110 raft_consensus.cc:482] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:45:04.832182 31110 raft_consensus.cc:504] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } } peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } }
I20260811 01:45:04.832940 31110 leader_election.cc:288] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39a96547f8ec4bf08372b8ce43c159a7 (127.21.214.67:38341), 26467dcaea844348b216edcec09ec9eb (127.21.214.65:34917)
I20260811 01:45:04.833726 30921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1266b3d9244e4434b51809cd5ce79e58" candidate_uuid: "56802ed9c3784965b20f771a4840de0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39a96547f8ec4bf08372b8ce43c159a7" is_pre_election: true
I20260811 01:45:04.833986 30921 raft_consensus.cc:2454] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56802ed9c3784965b20f771a4840de0b in term 0.
I20260811 01:45:04.833992 30590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1266b3d9244e4434b51809cd5ce79e58" candidate_uuid: "56802ed9c3784965b20f771a4840de0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26467dcaea844348b216edcec09ec9eb" is_pre_election: true
I20260811 01:45:04.834403 30590 raft_consensus.cc:2454] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56802ed9c3784965b20f771a4840de0b in term 0.
I20260811 01:45:04.834597 30683 leader_election.cc:302] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39a96547f8ec4bf08372b8ce43c159a7, 56802ed9c3784965b20f771a4840de0b; no voters:
I20260811 01:45:04.834949 31110 raft_consensus.cc:2781] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:45:04.835083 31110 raft_consensus.cc:482] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:45:04.835196 31110 raft_consensus.cc:3037] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:04.838075 31110 raft_consensus.cc:504] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } } peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } }
I20260811 01:45:04.838758 31110 leader_election.cc:288] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [CANDIDATE]: Term 1 election: Requested vote from peers 39a96547f8ec4bf08372b8ce43c159a7 (127.21.214.67:38341), 26467dcaea844348b216edcec09ec9eb (127.21.214.65:34917)
I20260811 01:45:04.839435 30921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1266b3d9244e4434b51809cd5ce79e58" candidate_uuid: "56802ed9c3784965b20f771a4840de0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39a96547f8ec4bf08372b8ce43c159a7"
I20260811 01:45:04.839664 30921 raft_consensus.cc:3037] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:04.839519 30590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1266b3d9244e4434b51809cd5ce79e58" candidate_uuid: "56802ed9c3784965b20f771a4840de0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26467dcaea844348b216edcec09ec9eb"
I20260811 01:45:04.839859 30590 raft_consensus.cc:3037] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:04.842545 30590 raft_consensus.cc:2454] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56802ed9c3784965b20f771a4840de0b in term 1.
I20260811 01:45:04.842769 30921 raft_consensus.cc:2454] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56802ed9c3784965b20f771a4840de0b in term 1.
I20260811 01:45:04.843068 30683 leader_election.cc:302] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26467dcaea844348b216edcec09ec9eb, 56802ed9c3784965b20f771a4840de0b; no voters:
I20260811 01:45:04.843459 31110 raft_consensus.cc:2781] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:45:04.844095 31110 raft_consensus.cc:686] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [term 1 LEADER]: Becoming Leader. State: Replica: 56802ed9c3784965b20f771a4840de0b, State: Running, Role: LEADER
I20260811 01:45:04.845166 31110 consensus_queue.cc:237] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } } peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } }
I20260811 01:45:04.845424 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.844919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.66:59563)
0811 01:45:04.845089 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.845259 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 01:45:04.850106 30432 catalog_manager.cc:5495] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 56802ed9c3784965b20f771a4840de0b (127.21.214.66). New cstate: current_term: 1 leader_uuid: "56802ed9c3784965b20f771a4840de0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56802ed9c3784965b20f771a4840de0b" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 } health_report { overall_health: UNKNOWN } } }
I20260811 01:45:04.891108 31096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.885100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43270 (local address 127.21.214.66:38921)
0811 01:45:04.885373 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:04.885381 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:04.885492 (+ 111us) server_negotiation.cc:407] Connection header received
0811 01:45:04.885801 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:04.885805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:04.885905 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:04.886091 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:45:04.886498 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:04.887320 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:04.888789 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:04.889237 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:04.889402 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:04.889861 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:45:04.890480 (+ 619us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:45:04.890659 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:04.890728 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 01:45:04.890822 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260811 01:45:04.896396 30921 raft_consensus.cc:1261] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7 [term 1 FOLLOWER]: Refusing update from remote peer 56802ed9c3784965b20f771a4840de0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:45:04.896754 30590 raft_consensus.cc:1261] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb [term 1 FOLLOWER]: Refusing update from remote peer 56802ed9c3784965b20f771a4840de0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:45:04.897365 31112 consensus_queue.cc:1035] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "39a96547f8ec4bf08372b8ce43c159a7" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:45:04.897790 31110 consensus_queue.cc:1035] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "26467dcaea844348b216edcec09ec9eb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 34917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:45:04.900732 31077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.900114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.67:52161)
0811 01:45:04.900383 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.900498 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260811 01:45:04.901285 31076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.900636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.65:47229)
0811 01:45:04.900932 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.901051 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260811 01:45:04.912789 31077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.912142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.67:59837)
0811 01:45:04.912481 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.912585 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210}
I20260811 01:45:04.912736 31116 mvcc.cc:205] Tried to move back new op lower bound from 7317146439245979648 to 7317146439050719232. Current Snapshot: MvccSnapshot[applied={T|T < 7317146439245979648 or (T in {7317146439245979648})}]
I20260811 01:45:04.914702 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.912079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.66:46537)
0811 01:45:04.914399 (+ 2320us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.914502 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2188}
I20260811 01:45:04.922245 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.921555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.66:37749)
0811 01:45:04.921844 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.921966 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260811 01:45:04.924930 31076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.924315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.21.214.65:44079)
0811 01:45:04.924588 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:04.924706 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260811 01:45:04.935778 31118 mvcc.cc:205] Tried to move back new op lower bound from 7317146439245979648 to 7317146439050719232. Current Snapshot: MvccSnapshot[applied={T|T < 7317146439245979648}]
I20260811 01:45:04.937196 30433 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:45:04.937599 30433 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:45:04.947494 30433 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22b5c1f3bba6482d8e5d02f5b8b2497d: Sending DeleteTablet for 3 replicas of tablet 1266b3d9244e4434b51809cd5ce79e58
I20260811 01:45:04.948580 30899 tablet_service.cc:1514] Processing DeleteTablet for tablet 1266b3d9244e4434b51809cd5ce79e58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:04 UTC) from {username='oryx', principal='oryx/127.21.214.124@KRBTEST.COM'} at 127.0.0.1:49748
I20260811 01:45:04.948897 30737 tablet_service.cc:1514] Processing DeleteTablet for tablet 1266b3d9244e4434b51809cd5ce79e58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:04 UTC) from {username='oryx', principal='oryx/127.21.214.124@KRBTEST.COM'} at 127.0.0.1:43262
I20260811 01:45:04.949720 30568 tablet_service.cc:1514] Processing DeleteTablet for tablet 1266b3d9244e4434b51809cd5ce79e58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:04 UTC) from {username='oryx', principal='oryx/127.21.214.124@KRBTEST.COM'} at 127.0.0.1:52936
I20260811 01:45:04.952265 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 30398
I20260811 01:45:04.967468 31127 tablet_replica.cc:330] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7: stopping tablet replica
I20260811 01:45:04.967806 31127 raft_consensus.cc:2229] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:45:04.968201 31127 raft_consensus.cc:2258] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:04.972950 31128 tablet_replica.cc:330] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b: stopping tablet replica
I20260811 01:45:04.974700 31128 raft_consensus.cc:2229] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:45:04.977321 31128 raft_consensus.cc:2258] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:04.977675 31130 tablet_replica.cc:330] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb: stopping tablet replica
I20260811 01:45:04.978015 31130 raft_consensus.cc:2229] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:45:04.978423 31130 raft_consensus.cc:2258] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:04.979709 31127 ts_tablet_manager.cc:1905] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:04.979835 31128 ts_tablet_manager.cc:1905] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:04.981949 31130 ts_tablet_manager.cc:1905] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:04.986542 31127 ts_tablet_manager.cc:1918] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:45:04.986781 31127 log.cc:1196] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/1266b3d9244e4434b51809cd5ce79e58
I20260811 01:45:04.987368 31127 ts_tablet_manager.cc:1939] T 1266b3d9244e4434b51809cd5ce79e58 P 39a96547f8ec4bf08372b8ce43c159a7: Deleting consensus metadata
I20260811 01:45:04.988726 30415 catalog_manager.cc:4841] TS 39a96547f8ec4bf08372b8ce43c159a7 (127.21.214.67:38341): tablet 1266b3d9244e4434b51809cd5ce79e58 (table test-table [id=b942f9df28c144c08b25657d576a8642]) successfully deleted
I20260811 01:45:04.989271 31130 ts_tablet_manager.cc:1918] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:45:04.989449 31130 log.cc:1196] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/1266b3d9244e4434b51809cd5ce79e58
I20260811 01:45:04.989914 31130 ts_tablet_manager.cc:1939] T 1266b3d9244e4434b51809cd5ce79e58 P 26467dcaea844348b216edcec09ec9eb: Deleting consensus metadata
I20260811 01:45:04.991084 30415 catalog_manager.cc:4841] TS 26467dcaea844348b216edcec09ec9eb (127.21.214.65:34917): tablet 1266b3d9244e4434b51809cd5ce79e58 (table test-table [id=b942f9df28c144c08b25657d576a8642]) successfully deleted
I20260811 01:45:04.997742 31128 ts_tablet_manager.cc:1918] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:45:04.997921 31128 log.cc:1196] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/1266b3d9244e4434b51809cd5ce79e58
I20260811 01:45:04.998476 31128 ts_tablet_manager.cc:1939] T 1266b3d9244e4434b51809cd5ce79e58 P 56802ed9c3784965b20f771a4840de0b: Deleting consensus metadata
I20260811 01:45:04.999524 31099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:04.953704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52938 (local address 127.21.214.65:34917)
0811 01:45:04.957394 (+ 3690us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:04.957404 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:04.969024 (+ 11620us) server_negotiation.cc:407] Connection header received
0811 01:45:04.969494 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:04.969504 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:04.969628 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:04.969729 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:04.973325 (+ 3596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:04.974530 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:04.977754 (+ 3224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:04.978501 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:04.978622 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:04.987564 (+ 8942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:04.987587 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:04.987611 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:04.987664 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:04.992161 (+ 4497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:04.996508 (+ 4347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:04.996519 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:04.996523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:04.996739 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:04.997504 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:04.997509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:04.997512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:04.998017 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:04.998384 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:04.998819 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 01:45:04.999102 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3466}
I20260811 01:45:04.999804 30418 catalog_manager.cc:4841] TS 56802ed9c3784965b20f771a4840de0b (127.21.214.66:38921): tablet 1266b3d9244e4434b51809cd5ce79e58 (table test-table [id=b942f9df28c144c08b25657d576a8642]) successfully deleted
I20260811 01:45:05.000732 30639 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:05.091044 31098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:05.090422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:35936)
0811 01:45:05.090707 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:05.090848 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 01:45:05.092444 31098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:05.091935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:41815 (local address 127.0.0.1:35946)
0811 01:45:05.092128 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:05.092257 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:41815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 01:45:05.127530 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 30398
I20260811 01:45:05.157021 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 30665
I20260811 01:45:05.173825 31096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:05.157988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43276 (local address 127.21.214.66:38921)
0811 01:45:05.158255 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:05.158262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:05.158284 (+ 22us) server_negotiation.cc:407] Connection header received
0811 01:45:05.158366 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:05.158371 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:05.158478 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:05.158607 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:05.159929 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:05.161251 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:05.162541 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:05.163223 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:05.163356 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:05.166152 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:05.166169 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:05.166198 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:05.166241 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:05.170493 (+ 4252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:05.171339 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:05.171348 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:05.171351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:05.171522 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:05.171999 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:05.172005 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:05.172008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:05.172515 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:05.172816 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:05.173177 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 01:45:05.173416 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260811 01:45:05.174628 30809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:05.301460 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 30665
I20260811 01:45:05.327383 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 30829
I20260811 01:45:05.342335 31134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:05.328170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49764 (local address 127.21.214.67:38341)
0811 01:45:05.328695 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:05.328700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:05.328717 (+ 17us) server_negotiation.cc:407] Connection header received
0811 01:45:05.328787 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:05.328791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:05.328868 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:05.328999 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:05.330516 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:05.331231 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:05.332519 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:05.332996 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:05.333144 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:05.336054 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:05.336074 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:05.336092 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:05.336128 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:05.339173 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:05.339914 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:05.339929 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:05.339931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:05.340107 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:05.340558 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:05.340564 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:05.340568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:05.341017 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:05.341373 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:05.341717 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 01:45:05.341967 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":256,"threads_started":1}
I20260811 01:45:05.343088 30969 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:05.441614 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 30829
I20260811 01:45:05.465332 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 30038
I20260811 01:45:05.466831 30234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:05.588968 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 30038
I20260811 01:45:05.617126 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 30156
I20260811 01:45:05.618597 30473 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:05.747627 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 30156
2026-08-11T01:45:05Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (12619 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:45:05 dist-test-slave-9gcw krb5kdc[31141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:45:05 dist-test-slave-9gcw krb5kdc[31141](info): set up 2 sockets
Aug 11 01:45:05 dist-test-slave-9gcw krb5kdc[31141](info): commencing operation
krb5kdc: starting...
W20260811 01:45:07.057422 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.010s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:45:07 dist-test-slave-9gcw krb5kdc[31141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:45:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:45:07Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.126.keytab.
Entry for principal oryx/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.126.keytab.
I20260811 01:45:07.354065 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:44407
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.126.keytab
--principal=oryx/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:46371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:44407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:45:07.706297 31155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:07.706673 31155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:07.706857 31155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:07.714265 31155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:45:07.714438 31155 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 01:45:07.714542 31155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:07.714669 31155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:07.714775 31155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:45:07.714871 31155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:45:07.725059 31155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:46371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:44407
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.126.keytab
--principal=oryx/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:44407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:07.726522 31155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:07.728096 31155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:07.740365 31160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:07.740470 31163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:07.740494 31161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:07.741686 31155 server_base.cc:1034] running on GCE node
I20260811 01:45:07.742630 31155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:07.744242 31155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:07.745523 31155 hybrid_clock.cc:648] HybridClock initialized: now 1786412707745458 us; error 51 us; skew 500 ppm
I20260811 01:45:07.745922 31155 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:45:07 dist-test-slave-9gcw krb5kdc[31141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412707, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:45:07.749576 31155 init.cc:377] Logged in from keytab as oryx/127.21.214.126@KRBTEST.COM (short username oryx)
I20260811 01:45:07.751298 31155 webserver.cc:460] Webserver started at http://127.21.214.126:39071/ using document root <none> and password file <none>
I20260811 01:45:07.752113 31155 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:07.752233 31155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:07.752606 31155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:07.756000 31155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "f748157d97074d638cfbdcee4bf0369a"
format_stamp: "Formatted at 2026-08-11 01:45:07 on dist-test-slave-9gcw"
server_key: "160d850c40104b4c895579fb97e77a76"
server_key_iv: "b406f50f21a7004eebe8c08386259adc"
server_key_version: "encryptionkey@0"
I20260811 01:45:07.756896 31155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "f748157d97074d638cfbdcee4bf0369a"
format_stamp: "Formatted at 2026-08-11 01:45:07 on dist-test-slave-9gcw"
server_key: "160d850c40104b4c895579fb97e77a76"
server_key_iv: "b406f50f21a7004eebe8c08386259adc"
server_key_version: "encryptionkey@0"
I20260811 01:45:07.762691 31155 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 01:45:07.766440 31171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:07.767910 31155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 01:45:07.768131 31155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "f748157d97074d638cfbdcee4bf0369a"
format_stamp: "Formatted at 2026-08-11 01:45:07 on dist-test-slave-9gcw"
server_key: "160d850c40104b4c895579fb97e77a76"
server_key_iv: "b406f50f21a7004eebe8c08386259adc"
server_key_version: "encryptionkey@0"
I20260811 01:45:07.768378 31155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:07.783351 31155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:07.788214 31155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:07.788532 31155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:07.811896 31155 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:44407
I20260811 01:45:07.811903 31242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:44407 every 8 connection(s)
I20260811 01:45:07.813545 31155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:45:07.817615 31243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:07.822893 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 31155
I20260811 01:45:07.823127 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:45:07.823810 22361 external_mini_cluster.cc:1428] Setting key 3c27af266a3a6166a37f53d1bdcd505c
I20260811 01:45:07.829085 31243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a: Bootstrap starting.
I20260811 01:45:07.833205 31243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:07.834537 31243 log.cc:824] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a: Log is configured to *not* fsync() on all Append() calls
I20260811 01:45:07.838303 31243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a: No bootstrap required, opened a new log
Aug 11 01:45:07 dist-test-slave-9gcw krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM
I20260811 01:45:07.846946 31243 raft_consensus.cc:348] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f748157d97074d638cfbdcee4bf0369a" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44407 } }
I20260811 01:45:07.847404 31243 raft_consensus.cc:374] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:07.847540 31243 raft_consensus.cc:729] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f748157d97074d638cfbdcee4bf0369a, State: Initialized, Role: FOLLOWER
I20260811 01:45:07.848201 31243 consensus_queue.cc:260] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f748157d97074d638cfbdcee4bf0369a" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44407 } }
I20260811 01:45:07.848471 31243 raft_consensus.cc:388] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:45:07.848739 31243 raft_consensus.cc:482] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:45:07.848977 31243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:07.851039 31248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:07.825952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44956 (local address 127.21.214.126:44407)
0811 01:45:07.826905 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:07.826929 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:07.826989 (+ 60us) server_negotiation.cc:407] Connection header received
0811 01:45:07.828816 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:07.828841 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:07.829338 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:07.829680 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:07.831641 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:07.832955 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:07.834864 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:07.836101 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:07.836252 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:07.841716 (+ 5464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:07.841726 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:07.841745 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:07.841795 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:07.846090 (+ 4295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:07.847054 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:07.847061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:07.847069 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:07.847274 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:07.847800 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:07.847804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:07.847807 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:07.848468 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:07.848972 (+ 504us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:07.849327 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 01:45:07.849841 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":447,"threads_started":1}
I20260811 01:45:07.851801 31243 raft_consensus.cc:504] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f748157d97074d638cfbdcee4bf0369a" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44407 } }
I20260811 01:45:07.852366 31243 leader_election.cc:302] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f748157d97074d638cfbdcee4bf0369a; no voters:
I20260811 01:45:07.853197 31243 leader_election.cc:288] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:45:07.853343 31250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:45:07.855024 31250 raft_consensus.cc:686] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a [term 1 LEADER]: Becoming Leader. State: Replica: f748157d97074d638cfbdcee4bf0369a, State: Running, Role: LEADER
I20260811 01:45:07.855798 31243 sys_catalog.cc:564] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:45:07.855684 31250 consensus_queue.cc:237] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f748157d97074d638cfbdcee4bf0369a" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44407 } }
I20260811 01:45:07.859153 31251 sys_catalog.cc:455] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f748157d97074d638cfbdcee4bf0369a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f748157d97074d638cfbdcee4bf0369a" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44407 } } }
I20260811 01:45:07.859663 31251 sys_catalog.cc:458] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a [sys.catalog]: This master's current role is: LEADER
I20260811 01:45:07.860177 31253 sys_catalog.cc:455] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f748157d97074d638cfbdcee4bf0369a. Latest consensus state: current_term: 1 leader_uuid: "f748157d97074d638cfbdcee4bf0369a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f748157d97074d638cfbdcee4bf0369a" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44407 } } }
I20260811 01:45:07.860522 31253 sys_catalog.cc:458] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a [sys.catalog]: This master's current role is: LEADER
I20260811 01:45:07.860864 31263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:45:07.866585 31263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:45:07.876474 31263 catalog_manager.cc:1347] Generated new cluster ID: 2e4fe3e9d69b412dad983946e9cb453f
I20260811 01:45:07.876619 31263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:45:07.902722 31263 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:45:07.904852 31263 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:45:07.928876 31263 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a: Generated new TSK 0
I20260811 01:45:07.930083 31263 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.65.keytab.
Entry for principal oryx/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.65.keytab.
I20260811 01:45:07.996949 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:44407
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.65.keytab
--principal=oryx/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:46371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:45:08.377002 31274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:08.377477 31274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:08.377779 31274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:08.386003 31274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:08.386179 31274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:08.386345 31274 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 01:45:08.386490 31274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:45:08.396785 31274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:46371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.65.keytab
--principal=oryx/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.214.126:44407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:08.398309 31274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:08.399876 31274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:08.413434 31279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:08.413374 31281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:08.413401 31285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:08.414672 31274 server_base.cc:1034] running on GCE node
I20260811 01:45:08.415373 31274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:08.416481 31274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:08.417737 31274 hybrid_clock.cc:648] HybridClock initialized: now 1786412708417666 us; error 47 us; skew 500 ppm
I20260811 01:45:08.418059 31274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:45:08 dist-test-slave-9gcw krb5kdc[31141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412708, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:45:08.421553 31274 init.cc:377] Logged in from keytab as oryx/127.21.214.65@KRBTEST.COM (short username oryx)
I20260811 01:45:08.423094 31274 webserver.cc:460] Webserver started at http://127.21.214.65:43441/ using document root <none> and password file <none>
I20260811 01:45:08.423863 31274 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:08.424005 31274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:08.424371 31274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:08.427572 31274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "961b9bf90b0f45ed99618110bde72d37"
format_stamp: "Formatted at 2026-08-11 01:45:08 on dist-test-slave-9gcw"
server_key: "1e75e1320162eba0b09f723d38b86ce8"
server_key_iv: "0d22ef725fb103acd231af7b6c101d70"
server_key_version: "encryptionkey@0"
I20260811 01:45:08.428491 31274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "961b9bf90b0f45ed99618110bde72d37"
format_stamp: "Formatted at 2026-08-11 01:45:08 on dist-test-slave-9gcw"
server_key: "1e75e1320162eba0b09f723d38b86ce8"
server_key_iv: "0d22ef725fb103acd231af7b6c101d70"
server_key_version: "encryptionkey@0"
I20260811 01:45:08.434533 31274 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 01:45:08.438496 31292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:08.440120 31274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 01:45:08.440337 31274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "961b9bf90b0f45ed99618110bde72d37"
format_stamp: "Formatted at 2026-08-11 01:45:08 on dist-test-slave-9gcw"
server_key: "1e75e1320162eba0b09f723d38b86ce8"
server_key_iv: "0d22ef725fb103acd231af7b6c101d70"
server_key_version: "encryptionkey@0"
I20260811 01:45:08.440593 31274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:08.457628 31274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:08.462393 31274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:08.462709 31274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:08.465819 31274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:08.465935 31274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:08.466076 31274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:08.466157 31274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 01:45:08 dist-test-slave-9gcw krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412708, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM
I20260811 01:45:08.494565 31301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:08.466476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:39751 (local address 127.21.214.126:44407)
0811 01:45:08.467146 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:08.467150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:08.468320 (+ 1170us) server_negotiation.cc:407] Connection header received
0811 01:45:08.470548 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:08.470553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:08.470666 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:08.470799 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:08.473624 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:08.474905 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:08.478655 (+ 3750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:08.479394 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:08.479518 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:08.486090 (+ 6572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:08.486096 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:08.486099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:08.486136 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:08.489798 (+ 3662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:08.491045 (+ 1247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:08.491051 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:08.491055 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:08.491231 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:08.491888 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:08.491894 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:08.491897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:08.492294 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:08.492454 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:08.493709 (+ 1255us) server_negotiation.cc:299] Negotiation successful
0811 01:45:08.494046 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":445,"threads_started":1}
I20260811 01:45:08.496491 31302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:08.467228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:44407 (local address 127.21.214.65:39751)
0811 01:45:08.468115 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:08.468185 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:08.470234 (+ 2049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:08.471101 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:08.471115 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:08.472054 (+ 939us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:08.473329 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:08.473369 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:08.475203 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:08.475215 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:08.478414 (+ 3199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:08.478440 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:08.479983 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:08.479993 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:08.480454 (+ 461us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:08.481533 (+ 1079us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:08.481567 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:08.485713 (+ 4146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:08.489969 (+ 4256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:08.489980 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:08.490003 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:08.490812 (+ 809us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:08.491369 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:08.491378 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:08.491382 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:08.491752 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:08.492526 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:08.492533 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:08.493135 (+ 602us) client_negotiation.cc:769] Sending connection context
0811 01:45:08.493530 (+ 395us) client_negotiation.cc:240] Negotiation successful
0811 01:45:08.494311 (+ 781us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":390,"threads_started":1}
I20260811 01:45:08.517535 31274 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:42843
I20260811 01:45:08.517535 31436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:42843 every 8 connection(s)
I20260811 01:45:08.518906 31274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:45:08.519142 31301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:08.504530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:35079 (local address 127.21.214.126:44407)
0811 01:45:08.504741 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:08.504747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:08.505050 (+ 303us) server_negotiation.cc:407] Connection header received
0811 01:45:08.505376 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:08.505385 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:08.505452 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:08.505559 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:08.506561 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:08.507400 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:08.509538 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:08.510179 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:08.510355 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:08.512587 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:08.512596 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:08.512603 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:08.512653 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:08.515990 (+ 3337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:08.516922 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:08.516926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:08.516928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:08.517110 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:08.517543 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:08.517551 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:08.517556 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:08.517944 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:08.518120 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:08.518523 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 01:45:08.518761 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 01:45:08.519351 31403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:08.504123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:44407 (local address 127.21.214.65:35079)
0811 01:45:08.504912 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:08.504945 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:08.505135 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:08.505592 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:08.505597 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:08.506099 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:08.506406 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:08.506415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:08.507547 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:08.507552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:08.509393 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:08.509406 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:08.510636 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:08.510641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:08.510769 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:08.511607 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:08.511624 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:08.512373 (+ 749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:08.516145 (+ 3772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:08.516150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:08.516154 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:08.516752 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:08.517269 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:08.517276 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:08.517284 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:08.517417 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:08.518069 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:08.518074 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:08.518266 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 01:45:08.518416 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 01:45:08.518812 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":382,"threads_started":1}
I20260811 01:45:08.521375 31437 heartbeater.cc:347] Connected to a master server at 127.21.214.126:44407
I20260811 01:45:08.521690 31437 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:08.522547 31437 heartbeater.cc:511] Master 127.21.214.126:44407 requested a full tablet report, sending...
I20260811 01:45:08.524712 31191 ts_manager.cc:194] Registered new tserver with Master: 961b9bf90b0f45ed99618110bde72d37 (127.21.214.65:42843)
I20260811 01:45:08.526235 31191 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:39751
I20260811 01:45:08.527760 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 31274
I20260811 01:45:08.527956 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:45:08.528849 22361 external_mini_cluster.cc:1428] Setting key 345fcb182b48c18a9ab55817129246c2
WARNING: no policy specified for oryx/127.21.214.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.214.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.66.keytab.
Entry for principal oryx/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.66.keytab.
I20260811 01:45:08.589100 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:44407
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.66.keytab
--principal=oryx/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:46371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:45:08.948566 31441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:08.948928 31441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:08.949110 31441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:08.956784 31441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:08.956949 31441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:08.957094 31441 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 01:45:08.957300 31441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:45:08.969077 31441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:46371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.66.keytab
--principal=oryx/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.214.126:44407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:08.970808 31441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:08.972620 31441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:08.988062 31447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:08.988013 31450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:08.988667 31446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:08.989246 31441 server_base.cc:1034] running on GCE node
I20260811 01:45:08.989926 31441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:08.990936 31441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:08.992168 31441 hybrid_clock.cc:648] HybridClock initialized: now 1786412708992074 us; error 63 us; skew 500 ppm
I20260811 01:45:08.992506 31441 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:45:08 dist-test-slave-9gcw krb5kdc[31141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412708, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:45:08.996038 31441 init.cc:377] Logged in from keytab as oryx/127.21.214.66@KRBTEST.COM (short username oryx)
I20260811 01:45:08.997681 31441 webserver.cc:460] Webserver started at http://127.21.214.66:44083/ using document root <none> and password file <none>
I20260811 01:45:08.998490 31441 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:08.998660 31441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:08.999075 31441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:09.002508 31441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "ece61e18c26c469fac433ae77127b5b3"
format_stamp: "Formatted at 2026-08-11 01:45:08 on dist-test-slave-9gcw"
server_key: "413b41beb32239434d68a236a4fd3f15"
server_key_iv: "042163644d14b24f4601c757377acb5e"
server_key_version: "encryptionkey@0"
I20260811 01:45:09.003417 31441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "ece61e18c26c469fac433ae77127b5b3"
format_stamp: "Formatted at 2026-08-11 01:45:08 on dist-test-slave-9gcw"
server_key: "413b41beb32239434d68a236a4fd3f15"
server_key_iv: "042163644d14b24f4601c757377acb5e"
server_key_version: "encryptionkey@0"
I20260811 01:45:09.009542 31441 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 01:45:09.013479 31457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:09.014978 31441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:45:09.015162 31441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "ece61e18c26c469fac433ae77127b5b3"
format_stamp: "Formatted at 2026-08-11 01:45:08 on dist-test-slave-9gcw"
server_key: "413b41beb32239434d68a236a4fd3f15"
server_key_iv: "042163644d14b24f4601c757377acb5e"
server_key_version: "encryptionkey@0"
I20260811 01:45:09.015435 31441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:09.047574 31441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:09.052378 31441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:09.052662 31441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:09.055665 31441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:09.055853 31441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:09.056015 31441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:09.056152 31441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 01:45:09 dist-test-slave-9gcw krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412708, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM
I20260811 01:45:09.086304 31470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:09.056600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:48313 (local address 127.21.214.126:44407)
0811 01:45:09.059983 (+ 3383us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:09.059990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:09.060019 (+ 29us) server_negotiation.cc:407] Connection header received
0811 01:45:09.062254 (+ 2235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:09.062259 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:09.062396 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:09.062530 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:09.065535 (+ 3005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.066789 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.069824 (+ 3035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.070495 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.070647 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:09.077202 (+ 6555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:09.077213 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:09.077217 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:09.077267 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:09.081580 (+ 4313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:09.083081 (+ 1501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:09.083089 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:09.083093 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:09.083277 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:09.083952 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:09.083957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:09.083960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:09.084347 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:09.084515 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:09.085662 (+ 1147us) server_negotiation.cc:299] Negotiation successful
0811 01:45:09.085900 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3233,"thread_start_us":3107,"threads_started":1}
I20260811 01:45:09.087800 31471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:09.057777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:44407 (local address 127.21.214.66:48313)
0811 01:45:09.059065 (+ 1288us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:09.059132 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:09.061753 (+ 2621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:09.062919 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:09.062937 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:09.064032 (+ 1095us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:09.065254 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:09.065293 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.067022 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.067028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:09.069643 (+ 2615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:09.069661 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.070813 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.070822 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:09.071044 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:09.072025 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:09.072072 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:09.076805 (+ 4733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:09.081875 (+ 5070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:09.081890 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:09.081920 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:09.082878 (+ 958us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:09.083413 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:09.083419 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:09.083426 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:09.083764 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:09.084612 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:09.084620 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:09.085135 (+ 515us) client_negotiation.cc:769] Sending connection context
0811 01:45:09.085545 (+ 410us) client_negotiation.cc:240] Negotiation successful
0811 01:45:09.086194 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"thread_start_us":515,"threads_started":1}
I20260811 01:45:09.114018 31441 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:33397
I20260811 01:45:09.114048 31597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:33397 every 8 connection(s)
I20260811 01:45:09.114164 31470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:09.095950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:43927 (local address 127.21.214.126:44407)
0811 01:45:09.096231 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:09.096238 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:09.097406 (+ 1168us) server_negotiation.cc:407] Connection header received
0811 01:45:09.097698 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:09.097708 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:09.097827 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:09.097988 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:09.099825 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.101224 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.104316 (+ 3092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.104851 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.105032 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:09.106992 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:09.106997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:09.107000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:09.107033 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:09.110173 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:09.111237 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:09.111245 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:09.111250 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:09.111444 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:09.111913 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:09.111919 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:09.111922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:09.112279 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:09.112460 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:09.113463 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0811 01:45:09.113773 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 01:45:09.114566 31557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:09.095956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:44407 (local address 127.21.214.66:43927)
0811 01:45:09.097255 (+ 1299us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:09.097294 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:09.097486 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:09.098044 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:09.098052 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:09.098976 (+ 924us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:09.099616 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:09.099640 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.101435 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.101444 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:09.104089 (+ 2645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:09.104117 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.105110 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.105118 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:09.105313 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:09.106002 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:09.106022 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:09.106781 (+ 759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:09.110358 (+ 3577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:09.110364 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:09.110368 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:09.111111 (+ 743us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:09.111612 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:09.111620 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:09.111625 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:09.111791 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:09.112409 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:09.112415 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:09.112636 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 01:45:09.112839 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 01:45:09.113471 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1103,"thread_start_us":927,"threads_started":1}
I20260811 01:45:09.115759 31441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:45:09.118155 31598 heartbeater.cc:347] Connected to a master server at 127.21.214.126:44407
I20260811 01:45:09.118458 31598 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:09.119403 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 31441
I20260811 01:45:09.119423 31598 heartbeater.cc:511] Master 127.21.214.126:44407 requested a full tablet report, sending...
I20260811 01:45:09.119596 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:45:09.120204 22361 external_mini_cluster.cc:1428] Setting key 6b116b94990813696742881c8ed7153f
I20260811 01:45:09.121217 31191 ts_manager.cc:194] Registered new tserver with Master: ece61e18c26c469fac433ae77127b5b3 (127.21.214.66:33397)
I20260811 01:45:09.122439 31191 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:48313
WARNING: no policy specified for oryx/127.21.214.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.214.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.67.keytab.
Entry for principal oryx/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.67.keytab.
I20260811 01:45:09.182812 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:44407
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.67.keytab
--principal=oryx/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:46371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:45:09.529640 31437 heartbeater.cc:503] Master 127.21.214.126:44407 was elected leader, sending a full tablet report...
W20260811 01:45:09.540370 31602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:09.540704 31602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:09.540899 31602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:09.549079 31602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:09.549252 31602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:09.549424 31602 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 01:45:09.549573 31602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:45:09.559896 31602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:46371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.67.keytab
--principal=oryx/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.214.126:44407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:09.561396 31602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:09.562810 31602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:09.577283 31612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:09.577488 31614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:09.577864 31611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:09.578006 31602 server_base.cc:1034] running on GCE node
I20260811 01:45:09.578985 31602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:09.580056 31602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:09.581393 31602 hybrid_clock.cc:648] HybridClock initialized: now 1786412709581280 us; error 68 us; skew 500 ppm
I20260811 01:45:09.581786 31602 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:45:09 dist-test-slave-9gcw krb5kdc[31141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412709, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:45:09.585608 31602 init.cc:377] Logged in from keytab as oryx/127.21.214.67@KRBTEST.COM (short username oryx)
I20260811 01:45:09.587373 31602 webserver.cc:460] Webserver started at http://127.21.214.67:38695/ using document root <none> and password file <none>
I20260811 01:45:09.588192 31602 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:09.588340 31602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:09.588721 31602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:09.592235 31602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "76ec1f0d2ef64a60a7ebcb3157402019"
format_stamp: "Formatted at 2026-08-11 01:45:09 on dist-test-slave-9gcw"
server_key: "521c999f5b7961955c24f0f7e2f69395"
server_key_iv: "984931b67886c88617c7e2f5e787d2ca"
server_key_version: "encryptionkey@0"
I20260811 01:45:09.593225 31602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "76ec1f0d2ef64a60a7ebcb3157402019"
format_stamp: "Formatted at 2026-08-11 01:45:09 on dist-test-slave-9gcw"
server_key: "521c999f5b7961955c24f0f7e2f69395"
server_key_iv: "984931b67886c88617c7e2f5e787d2ca"
server_key_version: "encryptionkey@0"
I20260811 01:45:09.599399 31602 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 01:45:09.603453 31621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:09.604904 31602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 01:45:09.605096 31602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "76ec1f0d2ef64a60a7ebcb3157402019"
format_stamp: "Formatted at 2026-08-11 01:45:09 on dist-test-slave-9gcw"
server_key: "521c999f5b7961955c24f0f7e2f69395"
server_key_iv: "984931b67886c88617c7e2f5e787d2ca"
server_key_version: "encryptionkey@0"
I20260811 01:45:09.605391 31602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:09.636646 31602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:09.641718 31602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:09.642040 31602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:09.645083 31602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:09.645314 31602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 01:45:09.645483 31602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:09.645568 31602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 01:45:09 dist-test-slave-9gcw krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412709, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM
I20260811 01:45:09.674253 31630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:09.646018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:54341 (local address 127.21.214.126:44407)
0811 01:45:09.649285 (+ 3267us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:09.649290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:09.649312 (+ 22us) server_negotiation.cc:407] Connection header received
0811 01:45:09.651963 (+ 2651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:09.651972 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:09.652081 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:09.652188 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:09.655248 (+ 3060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.656438 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.660061 (+ 3623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.660819 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.660967 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:09.666243 (+ 5276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:09.666251 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:09.666256 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:09.666312 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:09.669861 (+ 3549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:09.671052 (+ 1191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:09.671060 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:09.671067 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:09.671288 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:09.671823 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:09.671832 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:09.671838 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:09.672395 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:09.672608 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:09.673596 (+ 988us) server_negotiation.cc:299] Negotiation successful
0811 01:45:09.673845 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3144,"thread_start_us":2990,"threads_started":1}
I20260811 01:45:09.675426 31631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:09.646897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:44407 (local address 127.21.214.67:54341)
0811 01:45:09.648139 (+ 1242us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:09.648221 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:09.651491 (+ 3270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:09.652572 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:09.652590 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:09.653653 (+ 1063us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:09.654973 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:09.655006 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.656658 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.656667 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:09.659827 (+ 3160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:09.659851 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.661094 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.661102 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:09.661402 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:09.662393 (+ 991us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:09.662439 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:09.665865 (+ 3426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:09.670022 (+ 4157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:09.670032 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:09.670055 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:09.670856 (+ 801us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:09.671459 (+ 603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:09.671464 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:09.671468 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:09.671679 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:09.672556 (+ 877us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:09.672563 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:09.673022 (+ 459us) client_negotiation.cc:769] Sending connection context
0811 01:45:09.673482 (+ 460us) client_negotiation.cc:240] Negotiation successful
0811 01:45:09.674094 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":868,"thread_start_us":570,"threads_started":1}
I20260811 01:45:09.700085 31630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:09.681998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:52893 (local address 127.21.214.126:44407)
0811 01:45:09.682260 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:09.682270 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:09.682804 (+ 534us) server_negotiation.cc:407] Connection header received
0811 01:45:09.682980 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:09.682989 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:09.683118 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:09.683279 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:09.684392 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.686078 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.687488 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.688264 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.688421 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:09.691426 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:09.691437 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:09.691446 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:09.691526 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:09.696031 (+ 4505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:09.697332 (+ 1301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:09.697341 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:09.697346 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:09.697539 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:09.698154 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:09.698160 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:09.698165 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:09.698482 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:09.698653 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:09.699330 (+ 677us) server_negotiation.cc:299] Negotiation successful
0811 01:45:09.699635 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 01:45:09.700539 31711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:09.682000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:44407 (local address 127.21.214.67:52893)
0811 01:45:09.682690 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:09.682715 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:09.682839 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:09.683298 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:09.683301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:09.683862 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:09.684212 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:09.684223 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.686253 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.686257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:09.687321 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:09.687333 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.688514 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.688522 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:09.688725 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:09.689567 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:09.689600 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:09.691099 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:09.696198 (+ 5099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:09.696205 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:09.696211 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:09.697077 (+ 866us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:09.697698 (+ 621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:09.697709 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:09.697715 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:09.697999 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:09.698640 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:09.698648 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:09.698956 (+ 308us) client_negotiation.cc:769] Sending connection context
0811 01:45:09.699161 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 01:45:09.699761 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":384,"threads_started":1}
I20260811 01:45:09.706794 31602 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:45041
I20260811 01:45:09.706831 31756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:45041 every 8 connection(s)
I20260811 01:45:09.708087 31602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:45:09.710397 31757 heartbeater.cc:347] Connected to a master server at 127.21.214.126:44407
I20260811 01:45:09.710638 31757 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:09.711301 31757 heartbeater.cc:511] Master 127.21.214.126:44407 requested a full tablet report, sending...
I20260811 01:45:09.711710 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 31602
I20260811 01:45:09.711846 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:45:09.712400 22361 external_mini_cluster.cc:1428] Setting key 7836b3b571534bbf760edaddc8dcb9bf
I20260811 01:45:09.712874 31191 ts_manager.cc:194] Registered new tserver with Master: 76ec1f0d2ef64a60a7ebcb3157402019 (127.21.214.67:45041)
I20260811 01:45:09.714077 31191 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:54341
I20260811 01:45:09.715571 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 01:45:09 dist-test-slave-9gcw krb5kdc[31141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412709, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 01:45:09 dist-test-slave-9gcw krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:09 dist-test-slave-9gcw krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 01:45:09.781335 31630 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:09.769301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44970 (local address 127.21.214.126:44407)
0811 01:45:09.769535 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:09.769543 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:09.769811 (+ 268us) server_negotiation.cc:407] Connection header received
0811 01:45:09.769973 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:09.769979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:09.770096 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:09.770294 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:09.771727 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.772607 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.774110 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.774617 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.774735 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:09.780824 (+ 6089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44970: BlockingRecv error: recv got EOF from 127.0.0.1:44970 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 01:45:09 dist-test-slave-9gcw krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412709, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM
I20260811 01:45:09.801123 31630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:09.785543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44972 (local address 127.21.214.126:44407)
0811 01:45:09.785826 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:09.785832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:09.786038 (+ 206us) server_negotiation.cc:407] Connection header received
0811 01:45:09.786176 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:09.786180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:09.786260 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:09.786355 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:09.788021 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.788780 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.790278 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.790746 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.790846 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:09.795041 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:09.795057 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:09.795062 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:09.795116 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:09.798086 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:09.798877 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:09.798882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:09.798884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:09.799029 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:09.799435 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:09.799440 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:09.799442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:09.800013 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:09.800269 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:09.800489 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 01:45:09.800734 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 01:45:09.805855 31190 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44972:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:45:09.813486 31190 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:45:09.842162 31785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:09.827260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:33397 (local address 127.0.0.1:55216)
0811 01:45:09.829847 (+ 2587us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:09.829896 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:09.830112 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:09.831226 (+ 1114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:09.831236 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:09.831280 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:09.831823 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:09.831839 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.834218 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.834227 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:09.837717 (+ 3490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:09.837741 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.840727 (+ 2986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.840736 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:09.841026 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:09.841101 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 01:45:09.841314 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 01:45:09.841604 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2300,"thread_start_us":3196,"threads_started":1}
I20260811 01:45:09.843397 31782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:09.827526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55216 (local address 127.21.214.66:33397)
0811 01:45:09.828260 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:09.828291 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:09.830040 (+ 1749us) server_negotiation.cc:407] Connection header received
0811 01:45:09.830431 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:09.830443 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:09.830713 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:09.830945 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:09.832081 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.833998 (+ 1917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.837970 (+ 3972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.839999 (+ 2029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.840154 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:09.840469 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:09.842697 (+ 2228us) server_negotiation.cc:299] Negotiation successful
0811 01:45:09.842958 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":1222,"threads_started":1}
I20260811 01:45:09.844395 31786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:09.828408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38278 (local address 127.21.214.67:45041)
0811 01:45:09.829443 (+ 1035us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:09.829454 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:09.831082 (+ 1628us) server_negotiation.cc:407] Connection header received
0811 01:45:09.831210 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:09.831221 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:09.831536 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:09.831809 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:09.833195 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.835676 (+ 2481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.839571 (+ 3895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.841916 (+ 2345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.842098 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:09.842420 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:09.843813 (+ 1393us) server_negotiation.cc:299] Negotiation successful
0811 01:45:09.844001 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":803,"thread_start_us":557,"threads_started":1}
I20260811 01:45:09.843644 31781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:09.827585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:45041 (local address 127.0.0.1:38278)
0811 01:45:09.830779 (+ 3194us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:09.830800 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:09.830933 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:09.832247 (+ 1314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:09.832255 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:09.832295 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:09.832869 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:09.832886 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.835929 (+ 3043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.835938 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:09.839288 (+ 3350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:09.839313 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.842650 (+ 3337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.842659 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:09.842898 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:09.842933 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 01:45:09.843041 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 01:45:09.843198 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3049,"thread_start_us":2272,"threads_started":1}
I20260811 01:45:09.848575 31677 tablet_service.cc:1467] Processing CreateTablet for tablet 0c3b632f610640a1848be5b0955f7020 (DEFAULT_TABLE table=test-table [id=78c15797188c446eb3172a8301255de6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:09.848575 31517 tablet_service.cc:1467] Processing CreateTablet for tablet 0c3b632f610640a1848be5b0955f7020 (DEFAULT_TABLE table=test-table [id=78c15797188c446eb3172a8301255de6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:09.850273 31517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c3b632f610640a1848be5b0955f7020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:09.850267 31677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c3b632f610640a1848be5b0955f7020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:09.854362 31787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:09.831075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:42843 (local address 127.0.0.1:41396)
0811 01:45:09.843043 (+ 11968us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:09.843066 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:09.843200 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:09.846280 (+ 3080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:09.846285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:09.846316 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:09.846660 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:09.846670 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.849188 (+ 2518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.849196 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:09.850892 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:09.850906 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.853445 (+ 2539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.853450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:09.853595 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:09.853631 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 01:45:09.853740 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 01:45:09.853919 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11795,"thread_start_us":12191,"threads_started":1}
I20260811 01:45:09.854363 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:09.831582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41396 (local address 127.21.214.65:42843)
0811 01:45:09.845548 (+ 13966us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:09.845558 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:09.845589 (+ 31us) server_negotiation.cc:407] Connection header received
0811 01:45:09.845705 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:09.845715 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:09.846022 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:09.846301 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:09.846863 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.848992 (+ 2129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.851080 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:09.852845 (+ 1765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:09.853015 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:09.853307 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:09.853860 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 01:45:09.854053 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13789,"thread_start_us":13628,"threads_started":1}
I20260811 01:45:09.859104 31355 tablet_service.cc:1467] Processing CreateTablet for tablet 0c3b632f610640a1848be5b0955f7020 (DEFAULT_TABLE table=test-table [id=78c15797188c446eb3172a8301255de6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:09.860646 31355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c3b632f610640a1848be5b0955f7020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:09.866487 31789 tablet_bootstrap.cc:492] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019: Bootstrap starting.
I20260811 01:45:09.867424 31791 tablet_bootstrap.cc:492] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3: Bootstrap starting.
I20260811 01:45:09.871709 31791 tablet_bootstrap.cc:654] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:09.872357 31789 tablet_bootstrap.cc:654] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:09.873370 31791 log.cc:824] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3: Log is configured to *not* fsync() on all Append() calls
I20260811 01:45:09.873775 31789 log.cc:824] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019: Log is configured to *not* fsync() on all Append() calls
I20260811 01:45:09.876947 31794 tablet_bootstrap.cc:492] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37: Bootstrap starting.
I20260811 01:45:09.877095 31791 tablet_bootstrap.cc:492] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3: No bootstrap required, opened a new log
I20260811 01:45:09.877283 31789 tablet_bootstrap.cc:492] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019: No bootstrap required, opened a new log
I20260811 01:45:09.877489 31791 ts_tablet_manager.cc:1397] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.006s
I20260811 01:45:09.877616 31789 ts_tablet_manager.cc:1397] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 01:45:09.880942 31794 tablet_bootstrap.cc:654] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:09.882642 31794 log.cc:824] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37: Log is configured to *not* fsync() on all Append() calls
I20260811 01:45:09.886317 31794 tablet_bootstrap.cc:492] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37: No bootstrap required, opened a new log
I20260811 01:45:09.886636 31794 ts_tablet_manager.cc:1397] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260811 01:45:09.887524 31789 raft_consensus.cc:348] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } }
I20260811 01:45:09.887943 31789 raft_consensus.cc:374] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:09.888087 31789 raft_consensus.cc:729] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76ec1f0d2ef64a60a7ebcb3157402019, State: Initialized, Role: FOLLOWER
I20260811 01:45:09.888161 31791 raft_consensus.cc:348] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } }
I20260811 01:45:09.888639 31791 raft_consensus.cc:374] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:09.888747 31791 raft_consensus.cc:729] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece61e18c26c469fac433ae77127b5b3, State: Initialized, Role: FOLLOWER
I20260811 01:45:09.888845 31789 consensus_queue.cc:260] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } }
I20260811 01:45:09.889374 31791 consensus_queue.cc:260] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } }
I20260811 01:45:09.890239 31757 heartbeater.cc:503] Master 127.21.214.126:44407 was elected leader, sending a full tablet report...
I20260811 01:45:09.892540 31789 ts_tablet_manager.cc:1428] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
I20260811 01:45:09.898012 31598 heartbeater.cc:503] Master 127.21.214.126:44407 was elected leader, sending a full tablet report...
I20260811 01:45:09.898518 31791 ts_tablet_manager.cc:1428] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3: Time spent starting tablet: real 0.021s user 0.007s sys 0.013s
I20260811 01:45:09.900553 31794 raft_consensus.cc:348] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } }
I20260811 01:45:09.901135 31794 raft_consensus.cc:374] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:09.901328 31794 raft_consensus.cc:729] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 961b9bf90b0f45ed99618110bde72d37, State: Initialized, Role: FOLLOWER
I20260811 01:45:09.902118 31794 consensus_queue.cc:260] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } }
I20260811 01:45:09.904167 31794 ts_tablet_manager.cc:1428] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37: Time spent starting tablet: real 0.017s user 0.010s sys 0.006s
W20260811 01:45:09.960106 31758 tablet.cc:2369] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:45:10.022117 31438 tablet.cc:2369] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:45:10.118413 31599 tablet.cc:2369] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:45:10.119071 31799 raft_consensus.cc:482] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:45:10.119397 31799 raft_consensus.cc:504] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } }
I20260811 01:45:10.121642 31799 leader_election.cc:288] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece61e18c26c469fac433ae77127b5b3 (127.21.214.66:33397), 76ec1f0d2ef64a60a7ebcb3157402019 (127.21.214.67:45041)
I20260811 01:45:10.127384 31798 raft_consensus.cc:482] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:45:10.127701 31798 raft_consensus.cc:504] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } }
I20260811 01:45:10.129864 31798 leader_election.cc:288] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76ec1f0d2ef64a60a7ebcb3157402019 (127.21.214.67:45041), 961b9bf90b0f45ed99618110bde72d37 (127.21.214.65:42843)
I20260811 01:45:10.131145 31782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.122145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:58107 (local address 127.21.214.66:33397)
0811 01:45:10.122336 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:10.122343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:10.123924 (+ 1581us) server_negotiation.cc:407] Connection header received
0811 01:45:10.124118 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:10.124124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:10.124217 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:10.124277 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:10.125049 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.126263 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.128796 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.129741 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.129950 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.130027 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:10.130734 (+ 707us) server_negotiation.cc:299] Negotiation successful
0811 01:45:10.130877 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 01:45:10.131247 31786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.122634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:47681 (local address 127.21.214.67:45041)
0811 01:45:10.122839 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:10.122846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:10.123499 (+ 653us) server_negotiation.cc:407] Connection header received
0811 01:45:10.123833 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:10.123842 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:10.123931 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:10.124033 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:10.124636 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.125579 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.128839 (+ 3260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.130040 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.130226 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.130310 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:10.130821 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 01:45:10.130973 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 01:45:10.131954 31800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.122108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:33397 (local address 127.21.214.65:58107)
0811 01:45:10.123790 (+ 1682us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:10.123808 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:10.123945 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:10.124441 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:10.124444 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:10.124466 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:10.124859 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:10.124871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.126414 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.126420 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:10.128554 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:10.128569 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.130228 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.130238 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:10.130518 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.130556 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 01:45:10.131396 (+ 840us) client_negotiation.cc:240] Negotiation successful
0811 01:45:10.131542 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1539,"mutex_wait_us":38,"thread_start_us":882,"threads_started":1}
I20260811 01:45:10.133347 31540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3b632f610640a1848be5b0955f7020" candidate_uuid: "961b9bf90b0f45ed99618110bde72d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece61e18c26c469fac433ae77127b5b3" is_pre_election: true
I20260811 01:45:10.133941 31540 raft_consensus.cc:2454] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 961b9bf90b0f45ed99618110bde72d37 in term 0.
I20260811 01:45:10.134999 31294 leader_election.cc:302] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 961b9bf90b0f45ed99618110bde72d37, ece61e18c26c469fac433ae77127b5b3; no voters:
I20260811 01:45:10.135565 31799 raft_consensus.cc:2781] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:45:10.135742 31799 raft_consensus.cc:482] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:45:10.135865 31799 raft_consensus.cc:3037] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:10.136812 31801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.121962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:45041 (local address 127.21.214.65:47681)
0811 01:45:10.123357 (+ 1395us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:10.123380 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:10.123634 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:10.124066 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:10.124070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:10.124107 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:10.124469 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:10.124480 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.125718 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.125726 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:10.128546 (+ 2820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:10.128560 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.130292 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.130298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:10.130614 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.130651 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 01:45:10.136189 (+ 5538us) client_negotiation.cc:240] Negotiation successful
0811 01:45:10.136373 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1227,"mutex_wait_us":962,"thread_start_us":1511,"threads_started":1}
I20260811 01:45:10.138095 31700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3b632f610640a1848be5b0955f7020" candidate_uuid: "961b9bf90b0f45ed99618110bde72d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" is_pre_election: true
I20260811 01:45:10.138629 31700 raft_consensus.cc:2454] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 961b9bf90b0f45ed99618110bde72d37 in term 0.
I20260811 01:45:10.140149 31799 raft_consensus.cc:504] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } }
I20260811 01:45:10.140801 31786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.130514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:41957 (local address 127.21.214.67:45041)
0811 01:45:10.131490 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:10.131495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:10.131513 (+ 18us) server_negotiation.cc:407] Connection header received
0811 01:45:10.131580 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:10.131586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:10.131673 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:10.131767 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:10.132628 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.133855 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.136710 (+ 2855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.139479 (+ 2769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.139644 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.139723 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:10.140329 (+ 606us) server_negotiation.cc:299] Negotiation successful
0811 01:45:10.140471 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":883,"thread_start_us":5512,"threads_started":1}
I20260811 01:45:10.141108 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.131441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:53147 (local address 127.21.214.65:42843)
0811 01:45:10.131668 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:10.131674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:10.132043 (+ 369us) server_negotiation.cc:407] Connection header received
0811 01:45:10.132446 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:10.132451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:10.132542 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:10.132604 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:10.133491 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.134529 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.137976 (+ 3447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.139315 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.139924 (+ 609us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.140009 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:10.140717 (+ 708us) server_negotiation.cc:299] Negotiation successful
0811 01:45:10.140886 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 01:45:10.141218 31804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.131065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:42843 (local address 127.21.214.66:53147)
0811 01:45:10.131953 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:10.131971 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:10.132241 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:10.132820 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:10.132825 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:10.132851 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:10.133295 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:10.133308 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.134819 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.134824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:10.137786 (+ 2962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:10.137800 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.140257 (+ 2457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.140263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:10.140475 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.140510 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 01:45:10.140665 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 01:45:10.140815 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"thread_start_us":618,"threads_started":1}
I20260811 01:45:10.141532 31802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.130046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:45041 (local address 127.21.214.66:41957)
0811 01:45:10.130829 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:10.130861 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:10.130989 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:10.132001 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:10.132008 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:10.132042 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:10.132464 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:10.132478 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.134046 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.134051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:10.136494 (+ 2443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:10.136512 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.139896 (+ 3384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.139902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:10.140083 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.140125 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 01:45:10.141035 (+ 910us) client_negotiation.cc:240] Negotiation successful
0811 01:45:10.141228 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":647,"threads_started":1}
I20260811 01:45:10.141470 31540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3b632f610640a1848be5b0955f7020" candidate_uuid: "961b9bf90b0f45ed99618110bde72d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece61e18c26c469fac433ae77127b5b3"
I20260811 01:45:10.141741 31540 raft_consensus.cc:3037] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:10.141988 31799 leader_election.cc:288] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [CANDIDATE]: Term 1 election: Requested vote from peers ece61e18c26c469fac433ae77127b5b3 (127.21.214.66:33397), 76ec1f0d2ef64a60a7ebcb3157402019 (127.21.214.67:45041)
I20260811 01:45:10.142210 31700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3b632f610640a1848be5b0955f7020" candidate_uuid: "ece61e18c26c469fac433ae77127b5b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" is_pre_election: true
I20260811 01:45:10.142467 31700 raft_consensus.cc:2454] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece61e18c26c469fac433ae77127b5b3 in term 0.
I20260811 01:45:10.142594 31377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3b632f610640a1848be5b0955f7020" candidate_uuid: "ece61e18c26c469fac433ae77127b5b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "961b9bf90b0f45ed99618110bde72d37" is_pre_election: true
I20260811 01:45:10.142643 31699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3b632f610640a1848be5b0955f7020" candidate_uuid: "961b9bf90b0f45ed99618110bde72d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ec1f0d2ef64a60a7ebcb3157402019"
I20260811 01:45:10.142922 31699 raft_consensus.cc:3037] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:10.143070 31377 raft_consensus.cc:2379] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ece61e18c26c469fac433ae77127b5b3 in current term 1: Already voted for candidate 961b9bf90b0f45ed99618110bde72d37 in this term.
I20260811 01:45:10.143108 31463 leader_election.cc:302] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76ec1f0d2ef64a60a7ebcb3157402019, ece61e18c26c469fac433ae77127b5b3; no voters:
I20260811 01:45:10.145340 31540 raft_consensus.cc:2454] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 961b9bf90b0f45ed99618110bde72d37 in term 1.
I20260811 01:45:10.145673 31798 raft_consensus.cc:2741] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260811 01:45:10.145805 31294 leader_election.cc:302] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 961b9bf90b0f45ed99618110bde72d37, ece61e18c26c469fac433ae77127b5b3; no voters:
I20260811 01:45:10.146000 31699 raft_consensus.cc:2454] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 961b9bf90b0f45ed99618110bde72d37 in term 1.
I20260811 01:45:10.146147 31799 raft_consensus.cc:2781] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:45:10.146755 31799 raft_consensus.cc:686] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [term 1 LEADER]: Becoming Leader. State: Replica: 961b9bf90b0f45ed99618110bde72d37, State: Running, Role: LEADER
I20260811 01:45:10.147550 31799 consensus_queue.cc:237] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } }
I20260811 01:45:10.152307 31190 catalog_manager.cc:5495] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 961b9bf90b0f45ed99618110bde72d37 (127.21.214.65). New cstate: current_term: 1 leader_uuid: "961b9bf90b0f45ed99618110bde72d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } health_report { overall_health: HEALTHY } } }
I20260811 01:45:10.224458 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.219218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41406 (local address 127.21.214.65:42843)
0811 01:45:10.219452 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:10.219457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:10.219477 (+ 20us) server_negotiation.cc:407] Connection header received
0811 01:45:10.219620 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:10.219624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:10.219778 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:10.219932 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:45:10.220302 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.221111 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.222406 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.222786 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.222877 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.223247 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:45:10.223816 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:45:10.223931 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:10.224038 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 01:45:10.224105 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 01:45:10.232992 31699 raft_consensus.cc:1261] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Refusing update from remote peer 961b9bf90b0f45ed99618110bde72d37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:45:10.232995 31540 raft_consensus.cc:1261] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Refusing update from remote peer 961b9bf90b0f45ed99618110bde72d37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:45:10.234102 31805 consensus_queue.cc:1035] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:45:10.234469 31799 consensus_queue.cc:1035] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:45:10.246546 31808 mvcc.cc:205] Tried to move back new op lower bound from 7317146461103779840 to 7317146460768067584. Current Snapshot: MvccSnapshot[applied={T|T < 7317146461103779840}]
I20260811 01:45:10.249300 31809 mvcc.cc:205] Tried to move back new op lower bound from 7317146461103779840 to 7317146460768067584. Current Snapshot: MvccSnapshot[applied={T|T < 7317146461103779840}]
I20260811 01:45:10.262832 31190 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:45:10.263090 31190 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:45:10.267238 31190 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a: Sending DeleteTablet for 3 replicas of tablet 0c3b632f610640a1848be5b0955f7020
I20260811 01:45:10.268613 31517 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c3b632f610640a1848be5b0955f7020 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:10 UTC) from {username='oryx', principal='oryx/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:55216
I20260811 01:45:10.268955 31677 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c3b632f610640a1848be5b0955f7020 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:10 UTC) from {username='oryx', principal='oryx/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:38278
I20260811 01:45:10.269037 31355 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c3b632f610640a1848be5b0955f7020 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:10 UTC) from {username='oryx', principal='oryx/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:41396
I20260811 01:45:10.273895 31190 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44972:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:45:10.275655 31190 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:45:10.285280 31818 tablet_replica.cc:330] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3: stopping tablet replica
I20260811 01:45:10.286032 31818 raft_consensus.cc:2229] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:45:10.286700 31818 raft_consensus.cc:2258] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:10.287393 31820 tablet_replica.cc:330] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37: stopping tablet replica
I20260811 01:45:10.288255 31820 raft_consensus.cc:2229] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:45:10.290444 31820 raft_consensus.cc:2258] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:10.289126 31676 tablet_service.cc:1467] Processing CreateTablet for tablet 20f375e96402415887235e2deba69f5b (DEFAULT_TABLE table=test-table [id=7e4e17d44b8b46f3bcca378a5edd8d48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:10.288352 31355 tablet_service.cc:1467] Processing CreateTablet for tablet 20f375e96402415887235e2deba69f5b (DEFAULT_TABLE table=test-table [id=7e4e17d44b8b46f3bcca378a5edd8d48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:10.291148 31676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20f375e96402415887235e2deba69f5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:10.291432 31355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20f375e96402415887235e2deba69f5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:10.297618 31818 ts_tablet_manager.cc:1905] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:10.299367 31517 tablet_service.cc:1467] Processing CreateTablet for tablet 20f375e96402415887235e2deba69f5b (DEFAULT_TABLE table=test-table [id=7e4e17d44b8b46f3bcca378a5edd8d48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:10.299542 31819 tablet_replica.cc:330] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019: stopping tablet replica
I20260811 01:45:10.300086 31819 raft_consensus.cc:2229] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:45:10.300653 31517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20f375e96402415887235e2deba69f5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:10.300637 31819 raft_consensus.cc:2258] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:10.301949 31820 ts_tablet_manager.cc:1905] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:10.302964 31819 ts_tablet_manager.cc:1905] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:10.310448 31820 ts_tablet_manager.cc:1918] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:45:10.310755 31820 log.cc:1196] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/0c3b632f610640a1848be5b0955f7020
I20260811 01:45:10.310876 31794 tablet_bootstrap.cc:492] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37: Bootstrap starting.
I20260811 01:45:10.311448 31820 ts_tablet_manager.cc:1939] T 0c3b632f610640a1848be5b0955f7020 P 961b9bf90b0f45ed99618110bde72d37: Deleting consensus metadata
I20260811 01:45:10.312726 31794 tablet_bootstrap.cc:654] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:10.313707 31818 ts_tablet_manager.cc:1918] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:45:10.313833 31175 catalog_manager.cc:4841] TS 961b9bf90b0f45ed99618110bde72d37 (127.21.214.65:42843): tablet 0c3b632f610640a1848be5b0955f7020 (table test-table [id=78c15797188c446eb3172a8301255de6]) successfully deleted
I20260811 01:45:10.314015 31818 log.cc:1196] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/0c3b632f610640a1848be5b0955f7020
I20260811 01:45:10.314570 31789 tablet_bootstrap.cc:492] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019: Bootstrap starting.
I20260811 01:45:10.314741 31818 ts_tablet_manager.cc:1939] T 0c3b632f610640a1848be5b0955f7020 P ece61e18c26c469fac433ae77127b5b3: Deleting consensus metadata
I20260811 01:45:10.316570 31789 tablet_bootstrap.cc:654] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:10.317281 31174 catalog_manager.cc:4841] TS ece61e18c26c469fac433ae77127b5b3 (127.21.214.66:33397): tablet 0c3b632f610640a1848be5b0955f7020 (table test-table [id=78c15797188c446eb3172a8301255de6]) successfully deleted
I20260811 01:45:10.319993 31791 tablet_bootstrap.cc:492] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3: Bootstrap starting.
I20260811 01:45:10.321813 31791 tablet_bootstrap.cc:654] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:10.322712 31789 tablet_bootstrap.cc:492] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019: No bootstrap required, opened a new log
I20260811 01:45:10.324779 31789 ts_tablet_manager.cc:1397] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 01:45:10.325847 31819 ts_tablet_manager.cc:1918] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:45:10.326068 31819 log.cc:1196] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/0c3b632f610640a1848be5b0955f7020
I20260811 01:45:10.326740 31819 ts_tablet_manager.cc:1939] T 0c3b632f610640a1848be5b0955f7020 P 76ec1f0d2ef64a60a7ebcb3157402019: Deleting consensus metadata
I20260811 01:45:10.326521 31789 raft_consensus.cc:348] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.327068 31789 raft_consensus.cc:374] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:10.327343 31789 raft_consensus.cc:729] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76ec1f0d2ef64a60a7ebcb3157402019, State: Initialized, Role: FOLLOWER
I20260811 01:45:10.327869 31789 consensus_queue.cc:260] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.329099 31175 catalog_manager.cc:4841] TS 76ec1f0d2ef64a60a7ebcb3157402019 (127.21.214.67:45041): tablet 0c3b632f610640a1848be5b0955f7020 (table test-table [id=78c15797188c446eb3172a8301255de6]) successfully deleted
I20260811 01:45:10.331099 31794 tablet_bootstrap.cc:492] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37: No bootstrap required, opened a new log
I20260811 01:45:10.331297 31794 ts_tablet_manager.cc:1397] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37: Time spent bootstrapping tablet: real 0.021s user 0.010s sys 0.007s
I20260811 01:45:10.331351 31789 ts_tablet_manager.cc:1428] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260811 01:45:10.332499 31794 raft_consensus.cc:348] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.332919 31794 raft_consensus.cc:374] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:10.333039 31794 raft_consensus.cc:729] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 961b9bf90b0f45ed99618110bde72d37, State: Initialized, Role: FOLLOWER
I20260811 01:45:10.333375 31794 consensus_queue.cc:260] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.335242 31794 ts_tablet_manager.cc:1428] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260811 01:45:10.336369 31791 tablet_bootstrap.cc:492] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3: No bootstrap required, opened a new log
I20260811 01:45:10.336561 31791 ts_tablet_manager.cc:1397] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.006s
I20260811 01:45:10.338204 31791 raft_consensus.cc:348] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.338442 31791 raft_consensus.cc:374] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:10.338546 31791 raft_consensus.cc:729] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece61e18c26c469fac433ae77127b5b3, State: Initialized, Role: FOLLOWER
I20260811 01:45:10.338840 31791 consensus_queue.cc:260] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.339489 31791 ts_tablet_manager.cc:1428] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 01:45:10.470232 31813 raft_consensus.cc:482] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:45:10.470512 31813 raft_consensus.cc:504] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.471572 31813 leader_election.cc:288] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece61e18c26c469fac433ae77127b5b3 (127.21.214.66:33397), 76ec1f0d2ef64a60a7ebcb3157402019 (127.21.214.67:45041)
I20260811 01:45:10.472242 31540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f375e96402415887235e2deba69f5b" candidate_uuid: "961b9bf90b0f45ed99618110bde72d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece61e18c26c469fac433ae77127b5b3" is_pre_election: true
I20260811 01:45:10.472354 31699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f375e96402415887235e2deba69f5b" candidate_uuid: "961b9bf90b0f45ed99618110bde72d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" is_pre_election: true
I20260811 01:45:10.472590 31699 raft_consensus.cc:2454] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 961b9bf90b0f45ed99618110bde72d37 in term 0.
I20260811 01:45:10.472569 31540 raft_consensus.cc:2454] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 961b9bf90b0f45ed99618110bde72d37 in term 0.
I20260811 01:45:10.473124 31295 leader_election.cc:302] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76ec1f0d2ef64a60a7ebcb3157402019, 961b9bf90b0f45ed99618110bde72d37; no voters:
I20260811 01:45:10.473528 31813 raft_consensus.cc:2781] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:45:10.473665 31813 raft_consensus.cc:482] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:45:10.473789 31813 raft_consensus.cc:3037] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:10.476107 31813 raft_consensus.cc:504] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.476795 31813 leader_election.cc:288] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [CANDIDATE]: Term 1 election: Requested vote from peers ece61e18c26c469fac433ae77127b5b3 (127.21.214.66:33397), 76ec1f0d2ef64a60a7ebcb3157402019 (127.21.214.67:45041)
I20260811 01:45:10.477469 31699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f375e96402415887235e2deba69f5b" candidate_uuid: "961b9bf90b0f45ed99618110bde72d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ec1f0d2ef64a60a7ebcb3157402019"
I20260811 01:45:10.477442 31540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f375e96402415887235e2deba69f5b" candidate_uuid: "961b9bf90b0f45ed99618110bde72d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece61e18c26c469fac433ae77127b5b3"
I20260811 01:45:10.477694 31540 raft_consensus.cc:3037] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:10.477705 31699 raft_consensus.cc:3037] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:10.480863 31699 raft_consensus.cc:2454] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 961b9bf90b0f45ed99618110bde72d37 in term 1.
I20260811 01:45:10.480862 31540 raft_consensus.cc:2454] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 961b9bf90b0f45ed99618110bde72d37 in term 1.
I20260811 01:45:10.481448 31294 leader_election.cc:302] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 961b9bf90b0f45ed99618110bde72d37, ece61e18c26c469fac433ae77127b5b3; no voters:
I20260811 01:45:10.481864 31813 raft_consensus.cc:2781] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:45:10.482146 31813 raft_consensus.cc:686] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [term 1 LEADER]: Becoming Leader. State: Replica: 961b9bf90b0f45ed99618110bde72d37, State: Running, Role: LEADER
I20260811 01:45:10.482527 31813 consensus_queue.cc:237] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.486106 31191 catalog_manager.cc:5495] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 961b9bf90b0f45ed99618110bde72d37 (127.21.214.65). New cstate: current_term: 1 leader_uuid: "961b9bf90b0f45ed99618110bde72d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } health_report { overall_health: UNKNOWN } } }
Aug 11 01:45:10 dist-test-slave-9gcw krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412707, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.126@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM
I20260811 01:45:10.558497 31832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.535748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:44407 (local address 127.0.0.1:44988)
0811 01:45:10.542550 (+ 6802us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:10.542587 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:10.542822 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:10.543991 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:10.543996 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:10.544930 (+ 934us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:10.545137 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:10.545220 (+ 83us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.546613 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.546620 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:10.547563 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:10.547575 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.548289 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.548295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:10.548515 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.549265 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:10.549283 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:10.551558 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:10.555716 (+ 4158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:10.555727 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:10.555735 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:10.556286 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:10.556720 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:10.556723 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:10.556725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:10.556904 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:10.557654 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:10.557661 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:10.557818 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 01:45:10.557954 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 01:45:10.558156 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6567,"thread_start_us":7189,"threads_started":1}
I20260811 01:45:10.558883 31831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.535750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44988 (local address 127.21.214.126:44407)
0811 01:45:10.543692 (+ 7942us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:10.543697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:10.543717 (+ 20us) server_negotiation.cc:407] Connection header received
0811 01:45:10.543768 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:10.543772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:10.543834 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:10.543956 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:10.545396 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.546482 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.547707 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.548120 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.548271 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.551821 (+ 3550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:10.551829 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:10.551834 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:10.551876 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:10.555546 (+ 3670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:10.556430 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:10.556436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:10.556440 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:10.556602 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:10.557099 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:10.557105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:10.557108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:10.557512 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:10.557702 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:10.558115 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 01:45:10.558361 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7793,"thread_start_us":7655,"threads_started":1}
I20260811 01:45:10.564646 31191 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:44988:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 01:45:10.566066 31191 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:45:10.573851 31355 tablet_service.cc:1467] Processing CreateTablet for tablet fdba6c4ba05e414dbbd2072f7fec1eb1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e99fe3c6d94045f48a7d69d78b9e2d08]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 01:45:10.574265 31517 tablet_service.cc:1467] Processing CreateTablet for tablet fdba6c4ba05e414dbbd2072f7fec1eb1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e99fe3c6d94045f48a7d69d78b9e2d08]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 01:45:10.574396 31676 tablet_service.cc:1467] Processing CreateTablet for tablet fdba6c4ba05e414dbbd2072f7fec1eb1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e99fe3c6d94045f48a7d69d78b9e2d08]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 01:45:10.574589 31355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdba6c4ba05e414dbbd2072f7fec1eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:10.574889 31517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdba6c4ba05e414dbbd2072f7fec1eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:10.574939 31676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdba6c4ba05e414dbbd2072f7fec1eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:10.582898 31791 tablet_bootstrap.cc:492] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3: Bootstrap starting.
I20260811 01:45:10.583145 31789 tablet_bootstrap.cc:492] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019: Bootstrap starting.
I20260811 01:45:10.584868 31791 tablet_bootstrap.cc:654] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:10.585074 31789 tablet_bootstrap.cc:654] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:10.586304 31794 tablet_bootstrap.cc:492] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37: Bootstrap starting.
I20260811 01:45:10.588263 31794 tablet_bootstrap.cc:654] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:10.588723 31791 tablet_bootstrap.cc:492] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3: No bootstrap required, opened a new log
I20260811 01:45:10.588917 31791 ts_tablet_manager.cc:1397] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260811 01:45:10.590492 31791 raft_consensus.cc:348] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.590863 31791 raft_consensus.cc:374] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:10.590970 31791 raft_consensus.cc:729] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece61e18c26c469fac433ae77127b5b3, State: Initialized, Role: FOLLOWER
I20260811 01:45:10.591325 31791 consensus_queue.cc:260] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.591951 31798 tablet_replica.cc:439] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } }
I20260811 01:45:10.592321 31798 tablet_replica.cc:442] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:45:10.593191 31791 ts_tablet_manager.cc:1428] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 01:45:10.609194 31789 tablet_bootstrap.cc:492] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019: No bootstrap required, opened a new log
I20260811 01:45:10.609651 31789 ts_tablet_manager.cc:1397] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.012s
I20260811 01:45:10.609776 31794 tablet_bootstrap.cc:492] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37: No bootstrap required, opened a new log
I20260811 01:45:10.609989 31794 ts_tablet_manager.cc:1397] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.008s
I20260811 01:45:10.610610 31789 raft_consensus.cc:348] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.610927 31789 raft_consensus.cc:374] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:10.611025 31789 raft_consensus.cc:729] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76ec1f0d2ef64a60a7ebcb3157402019, State: Initialized, Role: FOLLOWER
I20260811 01:45:10.611105 31794 raft_consensus.cc:348] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.611438 31794 raft_consensus.cc:374] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:10.611302 31789 consensus_queue.cc:260] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.611570 31794 raft_consensus.cc:729] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 961b9bf90b0f45ed99618110bde72d37, State: Initialized, Role: FOLLOWER
I20260811 01:45:10.612030 31789 ts_tablet_manager.cc:1428] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 01:45:10.611903 31794 consensus_queue.cc:260] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.611922 31797 tablet_replica.cc:439] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } }
I20260811 01:45:10.612393 31797 tablet_replica.cc:442] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:45:10.612489 31813 tablet_replica.cc:439] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } }
I20260811 01:45:10.612695 31794 ts_tablet_manager.cc:1428] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 01:45:10.612825 31813 tablet_replica.cc:442] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:45:10.711901 31797 raft_consensus.cc:482] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:45:10.712136 31797 raft_consensus.cc:504] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.714115 31797 leader_election.cc:288] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 961b9bf90b0f45ed99618110bde72d37 (127.21.214.65:42843), ece61e18c26c469fac433ae77127b5b3 (127.21.214.66:33397)
I20260811 01:45:10.720016 31813 raft_consensus.cc:482] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:45:10.720259 31813 raft_consensus.cc:504] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.721962 31540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdba6c4ba05e414dbbd2072f7fec1eb1" candidate_uuid: "961b9bf90b0f45ed99618110bde72d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece61e18c26c469fac433ae77127b5b3" is_pre_election: true
I20260811 01:45:10.722221 31540 raft_consensus.cc:2454] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 961b9bf90b0f45ed99618110bde72d37 in term 0.
I20260811 01:45:10.722705 31294 leader_election.cc:302] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 961b9bf90b0f45ed99618110bde72d37, ece61e18c26c469fac433ae77127b5b3; no voters:
I20260811 01:45:10.723057 31805 raft_consensus.cc:2781] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:45:10.723179 31805 raft_consensus.cc:482] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:45:10.723292 31805 raft_consensus.cc:3037] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:10.726670 31805 raft_consensus.cc:504] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.727830 31540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdba6c4ba05e414dbbd2072f7fec1eb1" candidate_uuid: "961b9bf90b0f45ed99618110bde72d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece61e18c26c469fac433ae77127b5b3"
I20260811 01:45:10.721113 31813 leader_election.cc:288] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece61e18c26c469fac433ae77127b5b3 (127.21.214.66:33397), 76ec1f0d2ef64a60a7ebcb3157402019 (127.21.214.67:45041)
I20260811 01:45:10.728222 31540 raft_consensus.cc:3037] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:10.729166 31838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.714580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:33397 (local address 127.21.214.67:36585)
0811 01:45:10.720928 (+ 6348us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:10.720945 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:10.721097 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:10.721947 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:10.721953 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:10.721978 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:10.722448 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:10.722462 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.724145 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.724150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:10.726262 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:10.726277 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.728381 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.728390 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:10.728565 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.728599 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 01:45:10.728692 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 01:45:10.728829 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6222,"threads_started":1}
I20260811 01:45:10.729907 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.714985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:36585 (local address 127.21.214.66:33397)
0811 01:45:10.721412 (+ 6427us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:10.721418 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:10.721437 (+ 19us) server_negotiation.cc:407] Connection header received
0811 01:45:10.721584 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:10.721590 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:10.721678 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:10.721763 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:10.722648 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.723974 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.726448 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.727847 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.728198 (+ 351us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.728287 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:10.729423 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0811 01:45:10.729591 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6306,"threads_started":1}
I20260811 01:45:10.731269 31540 raft_consensus.cc:2454] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 961b9bf90b0f45ed99618110bde72d37 in term 1.
I20260811 01:45:10.731739 31294 leader_election.cc:302] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 961b9bf90b0f45ed99618110bde72d37, ece61e18c26c469fac433ae77127b5b3; no voters:
I20260811 01:45:10.732059 31813 raft_consensus.cc:2781] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:45:10.732193 31813 raft_consensus.cc:686] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [term 1 LEADER]: Becoming Leader. State: Replica: 961b9bf90b0f45ed99618110bde72d37, State: Running, Role: LEADER
I20260811 01:45:10.732470 31813 consensus_queue.cc:237] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:10.734995 31699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdba6c4ba05e414dbbd2072f7fec1eb1" candidate_uuid: "961b9bf90b0f45ed99618110bde72d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" is_pre_election: true
I20260811 01:45:10.735298 31699 raft_consensus.cc:2454] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 961b9bf90b0f45ed99618110bde72d37 in term 0.
I20260811 01:45:10.735639 31805 leader_election.cc:288] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [CANDIDATE]: Term 1 election: Requested vote from peers ece61e18c26c469fac433ae77127b5b3 (127.21.214.66:33397), 76ec1f0d2ef64a60a7ebcb3157402019 (127.21.214.67:45041)
I20260811 01:45:10.735756 31538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdba6c4ba05e414dbbd2072f7fec1eb1" candidate_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece61e18c26c469fac433ae77127b5b3" is_pre_election: true
I20260811 01:45:10.736095 31538 raft_consensus.cc:2379] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 76ec1f0d2ef64a60a7ebcb3157402019 in current term 1: Already voted for candidate 961b9bf90b0f45ed99618110bde72d37 in this term.
I20260811 01:45:10.736768 31699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdba6c4ba05e414dbbd2072f7fec1eb1" candidate_uuid: "961b9bf90b0f45ed99618110bde72d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ec1f0d2ef64a60a7ebcb3157402019"
I20260811 01:45:10.736969 31699 raft_consensus.cc:3037] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:10.738423 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.714695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:48201 (local address 127.21.214.65:42843)
0811 01:45:10.714894 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:10.714900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:10.720504 (+ 5604us) server_negotiation.cc:407] Connection header received
0811 01:45:10.730507 (+ 10003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:10.730512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:10.730586 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:10.730669 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:10.731476 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.732554 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.735182 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.736834 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.736995 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.737093 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:10.737977 (+ 884us) server_negotiation.cc:299] Negotiation successful
0811 01:45:10.738125 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 01:45:10.739372 31837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.714337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:42843 (local address 127.21.214.67:48201)
0811 01:45:10.720363 (+ 6026us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:10.720387 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:10.730338 (+ 9951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:10.730851 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:10.730856 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:10.730881 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:10.731232 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:10.731242 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.732858 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.732865 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:10.734956 (+ 2091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:10.734972 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.737564 (+ 2592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.737570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:10.737765 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.737812 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 01:45:10.738818 (+ 1006us) client_negotiation.cc:240] Negotiation successful
0811 01:45:10.738973 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5869,"spinlock_wait_cycles":3328,"thread_start_us":5748,"threads_started":1}
I20260811 01:45:10.740159 31377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdba6c4ba05e414dbbd2072f7fec1eb1" candidate_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "961b9bf90b0f45ed99618110bde72d37" is_pre_election: true
I20260811 01:45:10.740866 31624 leader_election.cc:302] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76ec1f0d2ef64a60a7ebcb3157402019; no voters: 961b9bf90b0f45ed99618110bde72d37, ece61e18c26c469fac433ae77127b5b3
I20260811 01:45:10.742280 31699 raft_consensus.cc:2454] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 961b9bf90b0f45ed99618110bde72d37 in term 1.
I20260811 01:45:10.742533 31797 raft_consensus.cc:2726] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 01:45:10.743206 31799 tablet_replica.cc:439] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37: TxnStatusTablet state changed. Reason: New leader 961b9bf90b0f45ed99618110bde72d37. Latest consensus state: current_term: 1 leader_uuid: "961b9bf90b0f45ed99618110bde72d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } }
I20260811 01:45:10.743595 31799 tablet_replica.cc:442] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37: This TxnStatusTablet replica's current role is: LEADER
I20260811 01:45:10.744777 31191 catalog_manager.cc:5495] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 961b9bf90b0f45ed99618110bde72d37 (127.21.214.65). New cstate: current_term: 1 leader_uuid: "961b9bf90b0f45ed99618110bde72d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } health_report { overall_health: UNKNOWN } } }
I20260811 01:45:10.746956 31841 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 01:45:10.747270 31841 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 01:45:10.796679 31832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.788924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:42843 (local address 127.0.0.1:41418)
0811 01:45:10.789183 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:10.789416 (+ 233us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:10.789599 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:10.790194 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:10.790199 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:10.790213 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 01:45:10.790414 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:10.790426 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.792160 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.792166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:10.793829 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:10.793847 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.795130 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.795136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:10.795365 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.795458 (+ 93us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 01:45:10.796024 (+ 566us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 01:45:10.796050 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 01:45:10.796136 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 01:45:10.796313 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 01:45:10.797387 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.788744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41418 (local address 127.21.214.65:42843)
0811 01:45:10.789080 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:10.789090 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:10.789512 (+ 422us) server_negotiation.cc:407] Connection header received
0811 01:45:10.789826 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:10.789834 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:10.789963 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:10.790091 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:45:10.790588 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.791912 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.794015 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.794846 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.794990 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.795708 (+ 718us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:45:10.795877 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:45:10.795966 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:10.796941 (+ 975us) server_negotiation.cc:299] Negotiation successful
0811 01:45:10.797029 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 01:45:10.803171 31699 raft_consensus.cc:1261] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Refusing update from remote peer 961b9bf90b0f45ed99618110bde72d37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:45:10.803375 31538 raft_consensus.cc:1261] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Refusing update from remote peer 961b9bf90b0f45ed99618110bde72d37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:45:10.803865 31799 consensus_queue.cc:1035] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:45:10.804136 31813 consensus_queue.cc:1035] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:45:10.818210 31797 tablet_replica.cc:439] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019: TxnStatusTablet state changed. Reason: New leader 961b9bf90b0f45ed99618110bde72d37. Latest consensus state: current_term: 1 leader_uuid: "961b9bf90b0f45ed99618110bde72d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } }
I20260811 01:45:10.818655 31797 tablet_replica.cc:442] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:45:10.819934 31798 tablet_replica.cc:439] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3: TxnStatusTablet state changed. Reason: New leader 961b9bf90b0f45ed99618110bde72d37. Latest consensus state: current_term: 1 leader_uuid: "961b9bf90b0f45ed99618110bde72d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } }
I20260811 01:45:10.820281 31798 tablet_replica.cc:442] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:45:10.825135 31844 mvcc.cc:205] Tried to move back new op lower bound from 7317146463439867904 to 7317146463165788160. Current Snapshot: MvccSnapshot[applied={T|T < 7317146463439867904}]
I20260811 01:45:10.834259 31799 tablet_replica.cc:439] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "961b9bf90b0f45ed99618110bde72d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } }
I20260811 01:45:10.834576 31799 tablet_replica.cc:442] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37: This TxnStatusTablet replica's current role is: LEADER
I20260811 01:45:10.836687 31799 tablet_replica.cc:439] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "961b9bf90b0f45ed99618110bde72d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } }
I20260811 01:45:10.836961 31799 tablet_replica.cc:442] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 961b9bf90b0f45ed99618110bde72d37: This TxnStatusTablet replica's current role is: LEADER
I20260811 01:45:10.840593 31798 tablet_replica.cc:439] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "961b9bf90b0f45ed99618110bde72d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } }
I20260811 01:45:10.840938 31798 tablet_replica.cc:442] T fdba6c4ba05e414dbbd2072f7fec1eb1 P ece61e18c26c469fac433ae77127b5b3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:45:10.842360 31797 tablet_replica.cc:439] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "961b9bf90b0f45ed99618110bde72d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } } }
I20260811 01:45:10.842636 31797 tablet_replica.cc:442] T fdba6c4ba05e414dbbd2072f7fec1eb1 P 76ec1f0d2ef64a60a7ebcb3157402019: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 01:45:10.867581 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.856762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:43823 (local address 127.21.214.65:42843)
0811 01:45:10.856991 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:10.857002 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:10.861259 (+ 4257us) server_negotiation.cc:407] Connection header received
0811 01:45:10.861502 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:10.861507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:10.861576 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:10.861730 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:45:10.862359 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.863235 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.865478 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.865988 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.866095 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.866776 (+ 681us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:45:10.866896 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:45:10.866976 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:10.867199 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 01:45:10.867281 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":68}
I20260811 01:45:10.867868 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:10.856546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:42843 (local address 127.21.214.65:43823)
0811 01:45:10.861110 (+ 4564us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:10.861138 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:10.861351 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:10.861761 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:10.861774 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:10.861788 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 01:45:10.862154 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:10.862167 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.863381 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.863387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:10.865304 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:10.865321 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:10.866196 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:10.866204 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:10.866407 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:10.866491 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 01:45:10.867010 (+ 519us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 01:45:10.867028 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 01:45:10.867129 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 01:45:10.867281 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4372,"thread_start_us":4148,"threads_started":1}
I20260811 01:45:10.878605 31699 raft_consensus.cc:1261] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Refusing update from remote peer 961b9bf90b0f45ed99618110bde72d37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:45:10.878764 31540 raft_consensus.cc:1261] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Refusing update from remote peer 961b9bf90b0f45ed99618110bde72d37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:45:10.879303 31813 consensus_queue.cc:1035] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:45:10.879614 31799 consensus_queue.cc:1035] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:45:10.959052 31191 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:45:10.959432 31191 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:45:10.962987 31191 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a: Sending DeleteTablet for 3 replicas of tablet 20f375e96402415887235e2deba69f5b
I20260811 01:45:10.963929 31353 tablet_service.cc:1514] Processing DeleteTablet for tablet 20f375e96402415887235e2deba69f5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:10 UTC) from {username='oryx', principal='oryx/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:41396
I20260811 01:45:10.964088 31517 tablet_service.cc:1514] Processing DeleteTablet for tablet 20f375e96402415887235e2deba69f5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:10 UTC) from {username='oryx', principal='oryx/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:55216
I20260811 01:45:10.964788 31676 tablet_service.cc:1514] Processing DeleteTablet for tablet 20f375e96402415887235e2deba69f5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:10 UTC) from {username='oryx', principal='oryx/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:38278
I20260811 01:45:10.980361 31862 tablet_replica.cc:330] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37: stopping tablet replica
I20260811 01:45:10.980665 31862 raft_consensus.cc:2229] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:45:10.981182 31862 raft_consensus.cc:2258] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:10.982908 31862 ts_tablet_manager.cc:1905] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:10.987120 31863 tablet_replica.cc:330] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019: stopping tablet replica
I20260811 01:45:10.987360 31863 raft_consensus.cc:2229] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:45:10.987676 31863 raft_consensus.cc:2258] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:10.992691 31863 ts_tablet_manager.cc:1905] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:10.996173 31862 ts_tablet_manager.cc:1918] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 01:45:10.996369 31862 log.cc:1196] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/20f375e96402415887235e2deba69f5b
I20260811 01:45:10.997087 31862 ts_tablet_manager.cc:1939] T 20f375e96402415887235e2deba69f5b P 961b9bf90b0f45ed99618110bde72d37: Deleting consensus metadata
I20260811 01:45:10.998934 31175 catalog_manager.cc:4841] TS 961b9bf90b0f45ed99618110bde72d37 (127.21.214.65:42843): tablet 20f375e96402415887235e2deba69f5b (table test-table [id=7e4e17d44b8b46f3bcca378a5edd8d48]) successfully deleted
I20260811 01:45:10.999997 31861 tablet_replica.cc:330] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3: stopping tablet replica
I20260811 01:45:11.000303 31861 raft_consensus.cc:2229] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:45:11.000605 31861 raft_consensus.cc:2258] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:11.001904 31861 ts_tablet_manager.cc:1905] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:11.003018 31863 ts_tablet_manager.cc:1918] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 01:45:11.003207 31863 log.cc:1196] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/20f375e96402415887235e2deba69f5b
I20260811 01:45:11.003772 31863 ts_tablet_manager.cc:1939] T 20f375e96402415887235e2deba69f5b P 76ec1f0d2ef64a60a7ebcb3157402019: Deleting consensus metadata
I20260811 01:45:11.005213 31175 catalog_manager.cc:4841] TS 76ec1f0d2ef64a60a7ebcb3157402019 (127.21.214.67:45041): tablet 20f375e96402415887235e2deba69f5b (table test-table [id=7e4e17d44b8b46f3bcca378a5edd8d48]) successfully deleted
I20260811 01:45:11.008879 31861 ts_tablet_manager.cc:1918] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 01:45:11.009071 31861 log.cc:1196] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/20f375e96402415887235e2deba69f5b
I20260811 01:45:11.009696 31861 ts_tablet_manager.cc:1939] T 20f375e96402415887235e2deba69f5b P ece61e18c26c469fac433ae77127b5b3: Deleting consensus metadata
I20260811 01:45:11.010923 31174 catalog_manager.cc:4841] TS ece61e18c26c469fac433ae77127b5b3 (127.21.214.66:33397): tablet 20f375e96402415887235e2deba69f5b (table test-table [id=7e4e17d44b8b46f3bcca378a5edd8d48]) successfully deleted
Aug 11 01:45:11 dist-test-slave-9gcw krb5kdc[31141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 01:45:11 dist-test-slave-9gcw krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:11 dist-test-slave-9gcw krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 01:45:11.072623 31831 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:11.054069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44998 (local address 127.21.214.126:44407)
0811 01:45:11.054353 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:11.054365 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:11.054794 (+ 429us) server_negotiation.cc:407] Connection header received
0811 01:45:11.055040 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:11.055045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:11.055162 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:11.055274 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:11.057312 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:11.058243 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:11.060264 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:11.060794 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:11.060934 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:11.072269 (+ 11335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44998: BlockingRecv error: recv got EOF from 127.0.0.1:44998 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 01:45:11 dist-test-slave-9gcw krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM
I20260811 01:45:11.095082 31831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:11.076525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45002 (local address 127.21.214.126:44407)
0811 01:45:11.076850 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:11.076858 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:11.077245 (+ 387us) server_negotiation.cc:407] Connection header received
0811 01:45:11.077494 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:11.077501 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:11.077664 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:11.077775 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:11.079501 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:11.080345 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:11.082316 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:11.082785 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:11.082907 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:11.088133 (+ 5226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:11.088140 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:11.088144 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:11.088187 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:11.091599 (+ 3412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:11.092410 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:11.092415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:11.092418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:11.092575 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:11.093059 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:11.093065 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:11.093067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:11.093444 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:11.093820 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:11.094241 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 01:45:11.094485 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260811 01:45:11.099311 31191 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:45:11.100142 31191 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:45:11.107923 31353 tablet_service.cc:1467] Processing CreateTablet for tablet bdcaf0ea01e445f8b23f7001dbfd2b70 (DEFAULT_TABLE table=test-table [id=7857a769548e41b5934b786c7d94a2a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:11.108633 31353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdcaf0ea01e445f8b23f7001dbfd2b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:11.108585 31517 tablet_service.cc:1467] Processing CreateTablet for tablet bdcaf0ea01e445f8b23f7001dbfd2b70 (DEFAULT_TABLE table=test-table [id=7857a769548e41b5934b786c7d94a2a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:11.109040 31517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdcaf0ea01e445f8b23f7001dbfd2b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:11.113566 31676 tablet_service.cc:1467] Processing CreateTablet for tablet bdcaf0ea01e445f8b23f7001dbfd2b70 (DEFAULT_TABLE table=test-table [id=7857a769548e41b5934b786c7d94a2a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:11.114089 31676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdcaf0ea01e445f8b23f7001dbfd2b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:11.126462 31879 tablet_bootstrap.cc:492] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019: Bootstrap starting.
I20260811 01:45:11.128371 31877 tablet_bootstrap.cc:492] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3: Bootstrap starting.
I20260811 01:45:11.130358 31877 tablet_bootstrap.cc:654] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:11.130740 31879 tablet_bootstrap.cc:654] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:11.132082 31878 tablet_bootstrap.cc:492] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37: Bootstrap starting.
I20260811 01:45:11.134967 31878 tablet_bootstrap.cc:654] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:11.142390 31879 tablet_bootstrap.cc:492] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019: No bootstrap required, opened a new log
I20260811 01:45:11.142699 31879 ts_tablet_manager.cc:1397] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.008s
I20260811 01:45:11.142750 31877 tablet_bootstrap.cc:492] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3: No bootstrap required, opened a new log
I20260811 01:45:11.143004 31877 ts_tablet_manager.cc:1397] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.004s
I20260811 01:45:11.143934 31879 raft_consensus.cc:348] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:11.143998 31877 raft_consensus.cc:348] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:11.144294 31877 raft_consensus.cc:374] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:11.144366 31879 raft_consensus.cc:374] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:11.144511 31877 raft_consensus.cc:729] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece61e18c26c469fac433ae77127b5b3, State: Initialized, Role: FOLLOWER
I20260811 01:45:11.144526 31879 raft_consensus.cc:729] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76ec1f0d2ef64a60a7ebcb3157402019, State: Initialized, Role: FOLLOWER
I20260811 01:45:11.144941 31879 consensus_queue.cc:260] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:11.144950 31877 consensus_queue.cc:260] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:11.145780 31877 ts_tablet_manager.cc:1428] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 01:45:11.145920 31879 ts_tablet_manager.cc:1428] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 01:45:11.147311 31878 tablet_bootstrap.cc:492] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37: No bootstrap required, opened a new log
I20260811 01:45:11.147492 31878 ts_tablet_manager.cc:1397] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.003s
I20260811 01:45:11.148473 31878 raft_consensus.cc:348] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:11.148753 31878 raft_consensus.cc:374] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:11.148880 31878 raft_consensus.cc:729] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 961b9bf90b0f45ed99618110bde72d37, State: Initialized, Role: FOLLOWER
I20260811 01:45:11.149205 31878 consensus_queue.cc:260] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:11.149873 31878 ts_tablet_manager.cc:1428] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 01:45:11.374300 31859 raft_consensus.cc:482] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:45:11.374574 31859 raft_consensus.cc:504] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:11.375442 31859 leader_election.cc:288] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece61e18c26c469fac433ae77127b5b3 (127.21.214.66:33397), 76ec1f0d2ef64a60a7ebcb3157402019 (127.21.214.67:45041)
I20260811 01:45:11.376010 31538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdcaf0ea01e445f8b23f7001dbfd2b70" candidate_uuid: "961b9bf90b0f45ed99618110bde72d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece61e18c26c469fac433ae77127b5b3" is_pre_election: true
I20260811 01:45:11.376022 31700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdcaf0ea01e445f8b23f7001dbfd2b70" candidate_uuid: "961b9bf90b0f45ed99618110bde72d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" is_pre_election: true
I20260811 01:45:11.376276 31700 raft_consensus.cc:2454] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 961b9bf90b0f45ed99618110bde72d37 in term 0.
I20260811 01:45:11.376291 31538 raft_consensus.cc:2454] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 961b9bf90b0f45ed99618110bde72d37 in term 0.
I20260811 01:45:11.376796 31294 leader_election.cc:302] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 961b9bf90b0f45ed99618110bde72d37, ece61e18c26c469fac433ae77127b5b3; no voters:
I20260811 01:45:11.377166 31859 raft_consensus.cc:2781] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:45:11.377358 31859 raft_consensus.cc:482] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:45:11.377439 31859 raft_consensus.cc:3037] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:11.379423 31859 raft_consensus.cc:504] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:11.379977 31859 leader_election.cc:288] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [CANDIDATE]: Term 1 election: Requested vote from peers ece61e18c26c469fac433ae77127b5b3 (127.21.214.66:33397), 76ec1f0d2ef64a60a7ebcb3157402019 (127.21.214.67:45041)
I20260811 01:45:11.380533 31538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdcaf0ea01e445f8b23f7001dbfd2b70" candidate_uuid: "961b9bf90b0f45ed99618110bde72d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece61e18c26c469fac433ae77127b5b3"
I20260811 01:45:11.380566 31700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdcaf0ea01e445f8b23f7001dbfd2b70" candidate_uuid: "961b9bf90b0f45ed99618110bde72d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ec1f0d2ef64a60a7ebcb3157402019"
I20260811 01:45:11.380774 31538 raft_consensus.cc:3037] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:11.380774 31700 raft_consensus.cc:3037] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:11.382936 31538 raft_consensus.cc:2454] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 961b9bf90b0f45ed99618110bde72d37 in term 1.
I20260811 01:45:11.382953 31700 raft_consensus.cc:2454] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 961b9bf90b0f45ed99618110bde72d37 in term 1.
I20260811 01:45:11.383477 31294 leader_election.cc:302] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 76ec1f0d2ef64a60a7ebcb3157402019, 961b9bf90b0f45ed99618110bde72d37, ece61e18c26c469fac433ae77127b5b3; no voters:
I20260811 01:45:11.383780 31859 raft_consensus.cc:2781] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:45:11.383958 31859 raft_consensus.cc:686] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [term 1 LEADER]: Becoming Leader. State: Replica: 961b9bf90b0f45ed99618110bde72d37, State: Running, Role: LEADER
I20260811 01:45:11.384279 31859 consensus_queue.cc:237] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:11.387075 31190 catalog_manager.cc:5495] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 961b9bf90b0f45ed99618110bde72d37 (127.21.214.65). New cstate: current_term: 1 leader_uuid: "961b9bf90b0f45ed99618110bde72d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } health_report { overall_health: UNKNOWN } } }
I20260811 01:45:11.427886 31886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:11.422127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41426 (local address 127.21.214.65:42843)
0811 01:45:11.422654 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:11.422659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:11.422676 (+ 17us) server_negotiation.cc:407] Connection header received
0811 01:45:11.422739 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:11.422743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:11.422809 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:11.422935 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:45:11.423369 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:11.424259 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:11.425874 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:11.426332 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:11.426480 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:11.427044 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:45:11.427177 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:45:11.427293 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:11.427466 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 01:45:11.427557 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":256,"threads_started":1}
I20260811 01:45:11.430706 31700 raft_consensus.cc:1261] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Refusing update from remote peer 961b9bf90b0f45ed99618110bde72d37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:45:11.430737 31538 raft_consensus.cc:1261] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Refusing update from remote peer 961b9bf90b0f45ed99618110bde72d37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:45:11.431710 31859 consensus_queue.cc:1035] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:45:11.432108 31813 consensus_queue.cc:1035] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:45:11.447726 31190 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:45:11.448017 31190 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:45:11.450944 31190 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a: Sending DeleteTablet for 3 replicas of tablet bdcaf0ea01e445f8b23f7001dbfd2b70
I20260811 01:45:11.451795 31353 tablet_service.cc:1514] Processing DeleteTablet for tablet bdcaf0ea01e445f8b23f7001dbfd2b70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:11 UTC) from {username='oryx', principal='oryx/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:41396
I20260811 01:45:11.451853 31676 tablet_service.cc:1514] Processing DeleteTablet for tablet bdcaf0ea01e445f8b23f7001dbfd2b70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:11 UTC) from {username='oryx', principal='oryx/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:38278
I20260811 01:45:11.451922 31517 tablet_service.cc:1514] Processing DeleteTablet for tablet bdcaf0ea01e445f8b23f7001dbfd2b70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:11 UTC) from {username='oryx', principal='oryx/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:55216
I20260811 01:45:11.452112 31862 tablet_replica.cc:330] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37: stopping tablet replica
I20260811 01:45:11.452185 31863 tablet_replica.cc:330] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019: stopping tablet replica
I20260811 01:45:11.452209 31861 tablet_replica.cc:330] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3: stopping tablet replica
I20260811 01:45:11.452337 31862 raft_consensus.cc:2229] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:45:11.452394 31863 raft_consensus.cc:2229] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:45:11.452410 31861 raft_consensus.cc:2229] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:45:11.452630 31863 raft_consensus.cc:2258] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:11.452627 31861 raft_consensus.cc:2258] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:11.452680 31862 raft_consensus.cc:2258] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:11.453696 31862 ts_tablet_manager.cc:1905] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:11.453495 31190 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 01:45:11.453819 31863 ts_tablet_manager.cc:1905] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:11.453961 31861 ts_tablet_manager.cc:1905] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 01:45:11.454296 31190 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:45:11.460680 31863 ts_tablet_manager.cc:1918] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:45:11.460909 31863 log.cc:1196] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/bdcaf0ea01e445f8b23f7001dbfd2b70
I20260811 01:45:11.461403 31863 ts_tablet_manager.cc:1939] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 76ec1f0d2ef64a60a7ebcb3157402019: Deleting consensus metadata
I20260811 01:45:11.461972 31861 ts_tablet_manager.cc:1918] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:45:11.462131 31861 log.cc:1196] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/bdcaf0ea01e445f8b23f7001dbfd2b70
I20260811 01:45:11.462568 31861 ts_tablet_manager.cc:1939] T bdcaf0ea01e445f8b23f7001dbfd2b70 P ece61e18c26c469fac433ae77127b5b3: Deleting consensus metadata
I20260811 01:45:11.462634 31175 catalog_manager.cc:4841] TS 76ec1f0d2ef64a60a7ebcb3157402019 (127.21.214.67:45041): tablet bdcaf0ea01e445f8b23f7001dbfd2b70 (table test-table [id=7857a769548e41b5934b786c7d94a2a2]) successfully deleted
I20260811 01:45:11.463632 31517 tablet_service.cc:1467] Processing CreateTablet for tablet 177182e42ada41d6b229a0231bd3be1f (DEFAULT_TABLE table=test-table [id=1322790c937d4aa2afbe59e432847679]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:11.463662 31353 tablet_service.cc:1467] Processing CreateTablet for tablet 177182e42ada41d6b229a0231bd3be1f (DEFAULT_TABLE table=test-table [id=1322790c937d4aa2afbe59e432847679]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:11.464098 31517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 177182e42ada41d6b229a0231bd3be1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:11.464113 31353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 177182e42ada41d6b229a0231bd3be1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:11.464612 31676 tablet_service.cc:1467] Processing CreateTablet for tablet 177182e42ada41d6b229a0231bd3be1f (DEFAULT_TABLE table=test-table [id=1322790c937d4aa2afbe59e432847679]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:11.465009 31676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 177182e42ada41d6b229a0231bd3be1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:11.465452 31862 ts_tablet_manager.cc:1918] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:45:11.465610 31862 log.cc:1196] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/bdcaf0ea01e445f8b23f7001dbfd2b70
I20260811 01:45:11.466095 31862 ts_tablet_manager.cc:1939] T bdcaf0ea01e445f8b23f7001dbfd2b70 P 961b9bf90b0f45ed99618110bde72d37: Deleting consensus metadata
I20260811 01:45:11.467247 31175 catalog_manager.cc:4841] TS 961b9bf90b0f45ed99618110bde72d37 (127.21.214.65:42843): tablet bdcaf0ea01e445f8b23f7001dbfd2b70 (table test-table [id=7857a769548e41b5934b786c7d94a2a2]) successfully deleted
I20260811 01:45:11.467491 31174 catalog_manager.cc:4841] TS ece61e18c26c469fac433ae77127b5b3 (127.21.214.66:33397): tablet bdcaf0ea01e445f8b23f7001dbfd2b70 (table test-table [id=7857a769548e41b5934b786c7d94a2a2]) successfully deleted
I20260811 01:45:11.472492 31879 tablet_bootstrap.cc:492] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019: Bootstrap starting.
I20260811 01:45:11.473664 31877 tablet_bootstrap.cc:492] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3: Bootstrap starting.
I20260811 01:45:11.474363 31879 tablet_bootstrap.cc:654] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:11.475850 31878 tablet_bootstrap.cc:492] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37: Bootstrap starting.
I20260811 01:45:11.476536 31877 tablet_bootstrap.cc:654] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:11.477717 31878 tablet_bootstrap.cc:654] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:11.480003 31879 tablet_bootstrap.cc:492] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019: No bootstrap required, opened a new log
I20260811 01:45:11.480208 31879 ts_tablet_manager.cc:1397] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 01:45:11.481032 31879 raft_consensus.cc:348] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:11.481403 31879 raft_consensus.cc:374] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:11.481532 31879 raft_consensus.cc:729] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76ec1f0d2ef64a60a7ebcb3157402019, State: Initialized, Role: FOLLOWER
I20260811 01:45:11.481858 31879 consensus_queue.cc:260] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:11.482499 31879 ts_tablet_manager.cc:1428] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 01:45:11.484620 31877 tablet_bootstrap.cc:492] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3: No bootstrap required, opened a new log
I20260811 01:45:11.484908 31877 ts_tablet_manager.cc:1397] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.004s
I20260811 01:45:11.486008 31877 raft_consensus.cc:348] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:11.486418 31877 raft_consensus.cc:374] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:11.486570 31877 raft_consensus.cc:729] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece61e18c26c469fac433ae77127b5b3, State: Initialized, Role: FOLLOWER
I20260811 01:45:11.486757 31878 tablet_bootstrap.cc:492] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37: No bootstrap required, opened a new log
I20260811 01:45:11.487006 31878 ts_tablet_manager.cc:1397] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 01:45:11.486956 31877 consensus_queue.cc:260] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:11.487787 31877 ts_tablet_manager.cc:1428] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 01:45:11.487865 31878 raft_consensus.cc:348] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:11.488157 31878 raft_consensus.cc:374] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:11.488278 31878 raft_consensus.cc:729] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 961b9bf90b0f45ed99618110bde72d37, State: Initialized, Role: FOLLOWER
I20260811 01:45:11.488545 31878 consensus_queue.cc:260] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:11.489166 31878 ts_tablet_manager.cc:1428] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 01:45:11.759686 31798 raft_consensus.cc:482] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:45:11.760015 31798 raft_consensus.cc:504] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:11.760890 31798 leader_election.cc:288] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 961b9bf90b0f45ed99618110bde72d37 (127.21.214.65:42843), 76ec1f0d2ef64a60a7ebcb3157402019 (127.21.214.67:45041)
I20260811 01:45:11.761677 31377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "177182e42ada41d6b229a0231bd3be1f" candidate_uuid: "ece61e18c26c469fac433ae77127b5b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "961b9bf90b0f45ed99618110bde72d37" is_pre_election: true
I20260811 01:45:11.761866 31700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "177182e42ada41d6b229a0231bd3be1f" candidate_uuid: "ece61e18c26c469fac433ae77127b5b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" is_pre_election: true
I20260811 01:45:11.762064 31377 raft_consensus.cc:2454] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece61e18c26c469fac433ae77127b5b3 in term 0.
I20260811 01:45:11.762146 31700 raft_consensus.cc:2454] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece61e18c26c469fac433ae77127b5b3 in term 0.
I20260811 01:45:11.762604 31463 leader_election.cc:302] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 961b9bf90b0f45ed99618110bde72d37, ece61e18c26c469fac433ae77127b5b3; no voters:
I20260811 01:45:11.762915 31798 raft_consensus.cc:2781] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:45:11.763062 31798 raft_consensus.cc:482] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:45:11.763165 31798 raft_consensus.cc:3037] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:11.765533 31798 raft_consensus.cc:504] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:11.766276 31798 leader_election.cc:288] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [CANDIDATE]: Term 1 election: Requested vote from peers 961b9bf90b0f45ed99618110bde72d37 (127.21.214.65:42843), 76ec1f0d2ef64a60a7ebcb3157402019 (127.21.214.67:45041)
I20260811 01:45:11.766782 31377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "177182e42ada41d6b229a0231bd3be1f" candidate_uuid: "ece61e18c26c469fac433ae77127b5b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "961b9bf90b0f45ed99618110bde72d37"
I20260811 01:45:11.766937 31700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "177182e42ada41d6b229a0231bd3be1f" candidate_uuid: "ece61e18c26c469fac433ae77127b5b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ec1f0d2ef64a60a7ebcb3157402019"
I20260811 01:45:11.767014 31377 raft_consensus.cc:3037] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:11.767166 31700 raft_consensus.cc:3037] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:11.769590 31700 raft_consensus.cc:2454] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece61e18c26c469fac433ae77127b5b3 in term 1.
I20260811 01:45:11.769618 31377 raft_consensus.cc:2454] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece61e18c26c469fac433ae77127b5b3 in term 1.
I20260811 01:45:11.770064 31463 leader_election.cc:302] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76ec1f0d2ef64a60a7ebcb3157402019, ece61e18c26c469fac433ae77127b5b3; no voters:
I20260811 01:45:11.770478 31798 raft_consensus.cc:2781] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:45:11.770892 31798 raft_consensus.cc:686] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [term 1 LEADER]: Becoming Leader. State: Replica: ece61e18c26c469fac433ae77127b5b3, State: Running, Role: LEADER
I20260811 01:45:11.771405 31798 consensus_queue.cc:237] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } }
I20260811 01:45:11.775098 31188 catalog_manager.cc:5495] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to ece61e18c26c469fac433ae77127b5b3 (127.21.214.66). New cstate: current_term: 1 leader_uuid: "ece61e18c26c469fac433ae77127b5b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece61e18c26c469fac433ae77127b5b3" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 } health_report { overall_health: UNKNOWN } } }
I20260811 01:45:11.861244 31895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:11.855116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55228 (local address 127.21.214.66:33397)
0811 01:45:11.855795 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:11.855800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:11.855814 (+ 14us) server_negotiation.cc:407] Connection header received
0811 01:45:11.855900 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:11.855903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:11.855986 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:11.856176 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:45:11.856618 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:11.857516 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:11.858915 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:11.859349 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:11.859486 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:11.859950 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:45:11.860552 (+ 602us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:45:11.860726 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:11.860789 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 01:45:11.860871 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":386,"threads_started":1}
I20260811 01:45:11.877305 31886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:11.871944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:44325 (local address 127.21.214.65:42843)
0811 01:45:11.872122 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:11.872127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:11.872363 (+ 236us) server_negotiation.cc:407] Connection header received
0811 01:45:11.872529 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:11.872536 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:11.872650 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:11.872758 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:45:11.873349 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:11.874109 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:11.875498 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:11.875925 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:11.876077 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:11.876516 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:45:11.876622 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:45:11.876725 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:11.876882 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 01:45:11.876964 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 01:45:11.877461 31898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:11.871551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:42843 (local address 127.21.214.66:44325)
0811 01:45:11.872248 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:11.872265 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:11.872393 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:11.872811 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:11.872818 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:11.872834 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 01:45:11.873096 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:11.873104 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:11.874260 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:11.874266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:11.875346 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:11.875359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:11.876114 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:11.876121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:11.876256 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:11.876322 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 01:45:11.876728 (+ 406us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 01:45:11.876751 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 01:45:11.876858 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 01:45:11.877019 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":381,"threads_started":1}
I20260811 01:45:11.886660 31700 raft_consensus.cc:1261] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Refusing update from remote peer ece61e18c26c469fac433ae77127b5b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:45:11.887055 31377 raft_consensus.cc:1261] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37 [term 1 FOLLOWER]: Refusing update from remote peer ece61e18c26c469fac433ae77127b5b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:45:11.887418 31892 consensus_queue.cc:1035] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76ec1f0d2ef64a60a7ebcb3157402019" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:45:11.887764 31798 consensus_queue.cc:1035] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "961b9bf90b0f45ed99618110bde72d37" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:45:11.924182 31895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:11.915482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:58727 (local address 127.21.214.66:33397)
0811 01:45:11.915824 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:11.915836 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:11.917872 (+ 2036us) server_negotiation.cc:407] Connection header received
0811 01:45:11.918196 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:11.918204 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:11.918296 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:11.918417 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:45:11.919012 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:11.920096 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:11.922183 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:11.922690 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:11.922900 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:11.923447 (+ 547us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:45:11.923587 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:45:11.923667 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:11.923879 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 01:45:11.923968 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260811 01:45:11.924649 31905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:11.915025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:33397 (local address 127.21.214.65:58727)
0811 01:45:11.917757 (+ 2732us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:11.917782 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:11.918018 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:11.918436 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:11.918441 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:11.918456 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 01:45:11.918794 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:11.918805 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:11.920254 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:11.920263 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:11.921984 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:11.922000 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:11.923033 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:11.923038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:11.923204 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:11.923251 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 01:45:11.923716 (+ 465us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 01:45:11.923732 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 01:45:11.923832 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 01:45:11.924002 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2517,"thread_start_us":2476,"threads_started":1}
I20260811 01:45:11.965600 31190 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:45:11.965903 31190 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:45:11.969280 31190 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f748157d97074d638cfbdcee4bf0369a: Sending DeleteTablet for 3 replicas of tablet 177182e42ada41d6b229a0231bd3be1f
I20260811 01:45:11.970358 31676 tablet_service.cc:1514] Processing DeleteTablet for tablet 177182e42ada41d6b229a0231bd3be1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:11 UTC) from {username='oryx', principal='oryx/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:38278
I20260811 01:45:11.970345 31517 tablet_service.cc:1514] Processing DeleteTablet for tablet 177182e42ada41d6b229a0231bd3be1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:11 UTC) from {username='oryx', principal='oryx/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:55216
I20260811 01:45:11.970592 31353 tablet_service.cc:1514] Processing DeleteTablet for tablet 177182e42ada41d6b229a0231bd3be1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:11 UTC) from {username='oryx', principal='oryx/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:41396
I20260811 01:45:11.972460 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 31274
I20260811 01:45:11.983562 31909 tablet_replica.cc:330] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019: stopping tablet replica
I20260811 01:45:11.984035 31909 raft_consensus.cc:2229] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:45:11.984444 31909 raft_consensus.cc:2258] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:11.985782 31909 ts_tablet_manager.cc:1905] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:11.988219 31910 tablet_replica.cc:330] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37: stopping tablet replica
I20260811 01:45:11.988554 31910 raft_consensus.cc:2229] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:45:11.988857 31910 raft_consensus.cc:2258] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:11.990245 31910 ts_tablet_manager.cc:1905] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:11.995404 31909 ts_tablet_manager.cc:1918] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 01:45:11.995692 31909 log.cc:1196] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/177182e42ada41d6b229a0231bd3be1f
I20260811 01:45:11.996371 31909 ts_tablet_manager.cc:1939] T 177182e42ada41d6b229a0231bd3be1f P 76ec1f0d2ef64a60a7ebcb3157402019: Deleting consensus metadata
I20260811 01:45:11.997411 31910 ts_tablet_manager.cc:1918] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 01:45:11.998109 31175 catalog_manager.cc:4841] TS 76ec1f0d2ef64a60a7ebcb3157402019 (127.21.214.67:45041): tablet 177182e42ada41d6b229a0231bd3be1f (table test-table [id=1322790c937d4aa2afbe59e432847679]) successfully deleted
I20260811 01:45:12.000036 31910 log.cc:1196] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/177182e42ada41d6b229a0231bd3be1f
I20260811 01:45:12.000049 31908 tablet_replica.cc:330] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3: stopping tablet replica
I20260811 01:45:12.000276 31908 raft_consensus.cc:2229] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:45:12.000620 31910 ts_tablet_manager.cc:1939] T 177182e42ada41d6b229a0231bd3be1f P 961b9bf90b0f45ed99618110bde72d37: Deleting consensus metadata
I20260811 01:45:12.000993 31908 raft_consensus.cc:2258] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:12.001915 31175 catalog_manager.cc:4841] TS 961b9bf90b0f45ed99618110bde72d37 (127.21.214.65:42843): tablet 177182e42ada41d6b229a0231bd3be1f (table test-table [id=1322790c937d4aa2afbe59e432847679]) successfully deleted
I20260811 01:45:12.002135 31908 ts_tablet_manager.cc:1905] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 01:45:12 dist-test-slave-9gcw krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.214.65@KRBTEST.COM
I20260811 01:45:12.009735 31908 ts_tablet_manager.cc:1918] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 01:45:12.009922 31908 log.cc:1196] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/177182e42ada41d6b229a0231bd3be1f
I20260811 01:45:12.010504 31908 ts_tablet_manager.cc:1939] T 177182e42ada41d6b229a0231bd3be1f P ece61e18c26c469fac433ae77127b5b3: Deleting consensus metadata
I20260811 01:45:12.011816 31174 catalog_manager.cc:4841] TS ece61e18c26c469fac433ae77127b5b3 (127.21.214.66:33397): tablet 177182e42ada41d6b229a0231bd3be1f (table test-table [id=1322790c937d4aa2afbe59e432847679]) successfully deleted
I20260811 01:45:12.013099 31886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:11.973972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41432 (local address 127.21.214.65:42843)
0811 01:45:11.974182 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:11.974189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:11.983828 (+ 9639us) server_negotiation.cc:407] Connection header received
0811 01:45:11.984854 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:11.984860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:11.984973 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:11.985059 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:11.989278 (+ 4219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:11.990382 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:11.999594 (+ 9212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:12.000421 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:12.000572 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:12.005941 (+ 5369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:12.005960 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:12.005976 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:12.006017 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:12.009297 (+ 3280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:12.010387 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:12.010399 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:12.010408 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:12.010687 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:12.011115 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:12.011121 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:12.011124 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:12.011546 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:12.011888 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:12.012618 (+ 730us) server_negotiation.cc:299] Negotiation successful
0811 01:45:12.012805 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 01:45:12.013912 31431 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:12.182348 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 31274
I20260811 01:45:12.229074 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 31441
Aug 11 01:45:12 dist-test-slave-9gcw krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.214.66@KRBTEST.COM
I20260811 01:45:12.245828 31895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:12.230021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55236 (local address 127.21.214.66:33397)
0811 01:45:12.230303 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:12.230310 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:12.230345 (+ 35us) server_negotiation.cc:407] Connection header received
0811 01:45:12.230426 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:12.230430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:12.230496 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:12.230629 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:12.231885 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:12.232678 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:12.234103 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:12.234592 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:12.234741 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:12.239312 (+ 4571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:12.239325 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:12.239340 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:12.239380 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:12.242653 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:12.243388 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:12.243396 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:12.243398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:12.243549 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:12.243962 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:12.243968 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:12.243971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:12.244360 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:12.244734 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:12.245045 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 01:45:12.245353 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260811 01:45:12.246555 31593 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:12.379060 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 31441
I20260811 01:45:12.406275 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 31602
Aug 11 01:45:12 dist-test-slave-9gcw krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.214.67@KRBTEST.COM
I20260811 01:45:12.423225 31914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:12.407210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60136 (local address 127.21.214.67:45041)
0811 01:45:12.407801 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:12.407806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:12.407824 (+ 18us) server_negotiation.cc:407] Connection header received
0811 01:45:12.407884 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:12.407887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:12.407956 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:12.408057 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:12.409487 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:12.410451 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:12.411685 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:12.412170 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:12.412270 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:12.416896 (+ 4626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:12.416911 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:12.416929 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:12.416971 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:12.420007 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:12.420712 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:12.420720 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:12.420722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:12.420860 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:12.421604 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:12.421608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:12.421610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:12.422050 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:12.422310 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:12.422719 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 01:45:12.422907 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":283,"threads_started":1}
I20260811 01:45:12.424124 31751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:12.532497 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 31602
I20260811 01:45:12.554720 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 31155
I20260811 01:45:12.556362 31238 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:12.677927 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 31155
2026-08-11T01:45:12Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6928 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:45:12 dist-test-slave-9gcw krb5kdc[31925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:45:12 dist-test-slave-9gcw krb5kdc[31925](info): set up 2 sockets
Aug 11 01:45:12 dist-test-slave-9gcw krb5kdc[31925](info): commencing operation
krb5kdc: starting...
W20260811 01:45:13.910569 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:45:14 dist-test-slave-9gcw krb5kdc[31925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:45:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:45:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:45:14.205662 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:43377
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:37675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:43377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:45:14.563465 31941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:14.563800 31941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:14.563897 31941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:14.571187 31941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:45:14.571308 31941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:14.571357 31941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:14.571411 31941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:45:14.571465 31941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:45:14.581703 31941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:37675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:43377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:43377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:14.583065 31941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:14.584384 31941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:14.597370 31946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:14.597401 31949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:14.597329 31947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:14.598742 31941 server_base.cc:1034] running on GCE node
I20260811 01:45:14.599596 31941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:14.601083 31941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:14.602360 31941 hybrid_clock.cc:648] HybridClock initialized: now 1786412714602289 us; error 63 us; skew 500 ppm
I20260811 01:45:14.602769 31941 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T01:45:14Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1995 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:45:14 dist-test-slave-9gcw krb5kdc[31955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:45:14 dist-test-slave-9gcw krb5kdc[31955](info): set up 2 sockets
Aug 11 01:45:14 dist-test-slave-9gcw krb5kdc[31955](info): commencing operation
krb5kdc: starting...
W20260811 01:45:15.928202 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:45:16 dist-test-slave-9gcw krb5kdc[31955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:45:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:45:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:45:16.216843 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:46459
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:44933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:46459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:45:16.564288 31971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:16.564595 31971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:16.564788 31971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:16.572074 31971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:45:16.572206 31971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:16.572305 31971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:16.572412 31971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:45:16.572476 31971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:45:16.582532 31971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:44933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:46459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:46459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:16.583968 31971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:16.585376 31971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:16.597886 31976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:16.597831 31977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:16.597922 31979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:16.598961 31971 server_base.cc:1034] running on GCE node
I20260811 01:45:16.599829 31971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:16.601361 31971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:16.602634 31971 hybrid_clock.cc:648] HybridClock initialized: now 1786412716602567 us; error 46 us; skew 500 ppm
I20260811 01:45:16.602974 31971 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:45:16 dist-test-slave-9gcw krb5kdc[31955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412716, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:45:16.611754 31971 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:45:16.613544 31971 webserver.cc:460] Webserver started at http://127.21.214.126:41467/ using document root <none> and password file <none>
I20260811 01:45:16.614292 31971 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:16.614423 31971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:16.614771 31971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:16.618060 31971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "fe6bb40aef3f423cab96d5f64d6836b8"
format_stamp: "Formatted at 2026-08-11 01:45:16 on dist-test-slave-9gcw"
server_key: "1adbd3bc59e95a63c913951afffbe85a"
server_key_iv: "9fb1e5fac0f88c6a59b0f61a00e237e6"
server_key_version: "encryptionkey@0"
I20260811 01:45:16.618857 31971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "fe6bb40aef3f423cab96d5f64d6836b8"
format_stamp: "Formatted at 2026-08-11 01:45:16 on dist-test-slave-9gcw"
server_key: "1adbd3bc59e95a63c913951afffbe85a"
server_key_iv: "9fb1e5fac0f88c6a59b0f61a00e237e6"
server_key_version: "encryptionkey@0"
I20260811 01:45:16.624780 31971 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 01:45:16.628772 31986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:16.630498 31971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:45:16.630715 31971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "fe6bb40aef3f423cab96d5f64d6836b8"
format_stamp: "Formatted at 2026-08-11 01:45:16 on dist-test-slave-9gcw"
server_key: "1adbd3bc59e95a63c913951afffbe85a"
server_key_iv: "9fb1e5fac0f88c6a59b0f61a00e237e6"
server_key_version: "encryptionkey@0"
I20260811 01:45:16.630954 31971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:16.649737 31971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:16.654247 31971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:16.654556 31971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:16.684092 31971 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:46459
I20260811 01:45:16.684116 32041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:46459 every 8 connection(s)
I20260811 01:45:16.685765 31971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:45:16.686419 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 31971
I20260811 01:45:16.686677 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:45:16.687476 22361 external_mini_cluster.cc:1428] Setting key 30f1f99673c37049e339bf30d5d1c270
I20260811 01:45:16.690850 32042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 01:45:16 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:45:16.704823 32042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8: Bootstrap starting.
I20260811 01:45:16.708752 32042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:16.710175 32042 log.cc:824] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8: Log is configured to *not* fsync() on all Append() calls
I20260811 01:45:16.714032 32042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8: No bootstrap required, opened a new log
I20260811 01:45:16.714288 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:16.689782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40290 (local address 127.21.214.126:46459)
0811 01:45:16.690623 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:16.690640 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:16.690694 (+ 54us) server_negotiation.cc:407] Connection header received
0811 01:45:16.693205 (+ 2511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:16.693240 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:16.693779 (+ 539us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:16.694303 (+ 524us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:16.695850 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:16.698070 (+ 2220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:16.699414 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:16.700501 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:16.700701 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:16.705927 (+ 5226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:16.705937 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:16.705953 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:16.705996 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:16.709365 (+ 3369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:16.710357 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:16.710368 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:16.710378 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:16.710569 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:16.711076 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:16.711081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:16.711084 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:16.711864 (+ 780us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:16.712098 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:16.712689 (+ 591us) server_negotiation.cc:299] Negotiation successful
0811 01:45:16.713102 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":362,"threads_started":1}
I20260811 01:45:16.722208 32042 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe6bb40aef3f423cab96d5f64d6836b8" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 46459 } }
I20260811 01:45:16.722597 32042 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:16.722716 32042 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe6bb40aef3f423cab96d5f64d6836b8, State: Initialized, Role: FOLLOWER
I20260811 01:45:16.723383 32042 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe6bb40aef3f423cab96d5f64d6836b8" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 46459 } }
I20260811 01:45:16.723636 32042 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:45:16.723773 32042 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:45:16.723968 32042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:16.726354 32042 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe6bb40aef3f423cab96d5f64d6836b8" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 46459 } }
I20260811 01:45:16.726951 32042 leader_election.cc:302] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe6bb40aef3f423cab96d5f64d6836b8; no voters:
I20260811 01:45:16.727593 32042 leader_election.cc:288] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:45:16.727766 32047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:45:16.729215 32047 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [term 1 LEADER]: Becoming Leader. State: Replica: fe6bb40aef3f423cab96d5f64d6836b8, State: Running, Role: LEADER
I20260811 01:45:16.729808 32047 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe6bb40aef3f423cab96d5f64d6836b8" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 46459 } }
I20260811 01:45:16.730213 32042 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:45:16.732903 32049 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe6bb40aef3f423cab96d5f64d6836b8. Latest consensus state: current_term: 1 leader_uuid: "fe6bb40aef3f423cab96d5f64d6836b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe6bb40aef3f423cab96d5f64d6836b8" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 46459 } } }
I20260811 01:45:16.732913 32048 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe6bb40aef3f423cab96d5f64d6836b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe6bb40aef3f423cab96d5f64d6836b8" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 46459 } } }
I20260811 01:45:16.733414 32049 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [sys.catalog]: This master's current role is: LEADER
I20260811 01:45:16.733503 32048 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [sys.catalog]: This master's current role is: LEADER
I20260811 01:45:16.738122 32059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:45:16.744313 32059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:45:16.755117 32059 catalog_manager.cc:1347] Generated new cluster ID: 050b6d21ec164af5b03fceb26a8e3473
I20260811 01:45:16.755291 32059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:45:16.781400 32059 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:45:16.782918 32059 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:45:16.790091 32059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8: Generated new TSK 0
I20260811 01:45:16.791030 32059 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
I20260811 01:45:16.876737 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:46459
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:44933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:45:17.250172 32077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:17.250500 32077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:17.250676 32077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:17.259123 32077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:17.259274 32077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:17.259509 32077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:45:17.270727 32077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:44933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:46459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:17.272157 32077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:17.273636 32077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:17.288101 32084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:17.288050 32087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:17.288018 32085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:17.294358 32077 server_base.cc:1034] running on GCE node
I20260811 01:45:17.295214 32077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:17.296377 32077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:17.297637 32077 hybrid_clock.cc:648] HybridClock initialized: now 1786412717297575 us; error 57 us; skew 500 ppm
I20260811 01:45:17.298074 32077 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:45:17 dist-test-slave-9gcw krb5kdc[31955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412717, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:45:17.301523 32077 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:45:17.303153 32077 webserver.cc:460] Webserver started at http://127.21.214.65:36127/ using document root <none> and password file <none>
I20260811 01:45:17.303920 32077 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:17.304072 32077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:17.304419 32077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:17.307607 32077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "87cada0f4ddb4a879c15ef5027eac685"
format_stamp: "Formatted at 2026-08-11 01:45:17 on dist-test-slave-9gcw"
server_key: "8628e0ace4a6c433a7c489a71d7f3caf"
server_key_iv: "8a69daf7187ffa528ca49f84cd76a802"
server_key_version: "encryptionkey@0"
I20260811 01:45:17.308478 32077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "87cada0f4ddb4a879c15ef5027eac685"
format_stamp: "Formatted at 2026-08-11 01:45:17 on dist-test-slave-9gcw"
server_key: "8628e0ace4a6c433a7c489a71d7f3caf"
server_key_iv: "8a69daf7187ffa528ca49f84cd76a802"
server_key_version: "encryptionkey@0"
I20260811 01:45:17.314073 32077 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 01:45:17.318048 32095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:17.319571 32077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:45:17.319766 32077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "87cada0f4ddb4a879c15ef5027eac685"
format_stamp: "Formatted at 2026-08-11 01:45:17 on dist-test-slave-9gcw"
server_key: "8628e0ace4a6c433a7c489a71d7f3caf"
server_key_iv: "8a69daf7187ffa528ca49f84cd76a802"
server_key_version: "encryptionkey@0"
I20260811 01:45:17.319980 32077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:17.336740 32077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:17.341411 32077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:17.341745 32077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:17.342752 32077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:45:17.344218 32077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:17.344324 32077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:17.344491 32077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:17.344569 32077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:17.379392 32077 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:45059
I20260811 01:45:17.379446 32217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:45059 every 8 connection(s)
I20260811 01:45:17.380791 32077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:45:17.387351 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 32077
I20260811 01:45:17.387578 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:45:17.388445 22361 external_mini_cluster.cc:1428] Setting key ac02ca86ce8cee198deea38d37551685
Aug 11 01:45:17 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412717, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:45:17.402871 32221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:17.383609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:37453 (local address 127.21.214.126:46459)
0811 01:45:17.384134 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:17.384141 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:17.385374 (+ 1233us) server_negotiation.cc:407] Connection header received
0811 01:45:17.387517 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:17.387525 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:17.387654 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:17.387770 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:17.390005 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:17.390845 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:17.392467 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:17.392881 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:17.393020 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:17.396659 (+ 3639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:17.396664 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:17.396668 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:17.396701 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:17.399401 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:17.400183 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:17.400187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:17.400190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:17.400319 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:17.400784 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:17.400788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:17.400790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:17.401184 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:17.401358 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:17.402338 (+ 980us) server_negotiation.cc:299] Negotiation successful
0811 01:45:17.402588 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":243,"threads_started":1}
I20260811 01:45:17.404402 32222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:17.384380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:37453)
0811 01:45:17.385194 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:17.385250 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:17.387122 (+ 1872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:17.388122 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:17.388135 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:17.388933 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:17.389825 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:17.389848 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:17.391010 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:17.391020 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:17.392315 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:17.392327 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:17.393129 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:17.393138 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:17.393414 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:17.394115 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:17.394157 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:17.396415 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:17.399558 (+ 3143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:17.399568 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:17.399586 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:17.400038 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:17.400435 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:17.400440 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:17.400444 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:17.400649 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:17.401365 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:17.401378 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:17.401888 (+ 510us) client_negotiation.cc:769] Sending connection context
0811 01:45:17.402162 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 01:45:17.402757 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":320,"threads_started":1}
I20260811 01:45:17.406190 32218 heartbeater.cc:347] Connected to a master server at 127.21.214.126:46459
I20260811 01:45:17.406513 32218 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:17.407351 32218 heartbeater.cc:511] Master 127.21.214.126:46459 requested a full tablet report, sending...
I20260811 01:45:17.410073 32003 ts_manager.cc:194] Registered new tserver with Master: 87cada0f4ddb4a879c15ef5027eac685 (127.21.214.65:45059)
I20260811 01:45:17.412519 32003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:37453
WARNING: no policy specified for kudu/127.21.214.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
I20260811 01:45:17.458024 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:46459
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:44933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:45:17.817005 32225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:17.817548 32225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:17.817718 32225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:17.825667 32225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:17.825835 32225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:17.826026 32225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:45:17.836970 32225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:44933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:46459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:17.838493 32225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:17.839991 32225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:17.854498 32233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:17.854682 32230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:17.854866 32231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:17.855191 32225 server_base.cc:1034] running on GCE node
I20260811 01:45:17.855978 32225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:17.856969 32225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:17.858184 32225 hybrid_clock.cc:648] HybridClock initialized: now 1786412717858153 us; error 72 us; skew 500 ppm
I20260811 01:45:17.858546 32225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:45:17 dist-test-slave-9gcw krb5kdc[31955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412717, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:45:17.861994 32225 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:45:17.863734 32225 webserver.cc:460] Webserver started at http://127.21.214.66:44731/ using document root <none> and password file <none>
I20260811 01:45:17.864518 32225 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:17.864670 32225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:17.865015 32225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:17.868383 32225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "97bdd61c13e648c4bfeef76e16877d85"
format_stamp: "Formatted at 2026-08-11 01:45:17 on dist-test-slave-9gcw"
server_key: "d992f960fe82092354a55f64cd6e2356"
server_key_iv: "f04f0433f522aeff846b9515243111eb"
server_key_version: "encryptionkey@0"
I20260811 01:45:17.869359 32225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "97bdd61c13e648c4bfeef76e16877d85"
format_stamp: "Formatted at 2026-08-11 01:45:17 on dist-test-slave-9gcw"
server_key: "d992f960fe82092354a55f64cd6e2356"
server_key_iv: "f04f0433f522aeff846b9515243111eb"
server_key_version: "encryptionkey@0"
I20260811 01:45:17.874962 32225 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 01:45:17.878885 32241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:17.880378 32225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 01:45:17.880569 32225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "97bdd61c13e648c4bfeef76e16877d85"
format_stamp: "Formatted at 2026-08-11 01:45:17 on dist-test-slave-9gcw"
server_key: "d992f960fe82092354a55f64cd6e2356"
server_key_iv: "f04f0433f522aeff846b9515243111eb"
server_key_version: "encryptionkey@0"
I20260811 01:45:17.880805 32225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:17.897130 32225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:17.901687 32225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:17.902007 32225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:17.903029 32225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:45:17.904553 32225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:17.904672 32225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:17.904824 32225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:17.904897 32225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:17.941509 32225 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:33005
I20260811 01:45:17.941597 32369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:33005 every 8 connection(s)
I20260811 01:45:17.942886 32225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:45:17.947666 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 32225
I20260811 01:45:17.947964 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:45:17.948540 22361 external_mini_cluster.cc:1428] Setting key f3b8d34ad4a823097e8f754ee744097c
Aug 11 01:45:17 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412717, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:45:17.970125 32372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:17.945718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:43615 (local address 127.21.214.126:46459)
0811 01:45:17.946238 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:17.946244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:17.947419 (+ 1175us) server_negotiation.cc:407] Connection header received
0811 01:45:17.949934 (+ 2515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:17.949938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:17.950012 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:17.950104 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:17.952244 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:17.952992 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:17.955548 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:17.956224 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:17.956362 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:17.961400 (+ 5038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:17.961407 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:17.961411 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:17.961453 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:17.965733 (+ 4280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:17.966863 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:17.966872 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:17.966879 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:17.967098 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:17.967624 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:17.967632 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:17.967639 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:17.968125 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:17.968304 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:17.969239 (+ 935us) server_negotiation.cc:299] Negotiation successful
0811 01:45:17.969550 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":252,"threads_started":1}
I20260811 01:45:17.970837 32374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:17.946350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:43615)
0811 01:45:17.947120 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:17.947250 (+ 130us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:17.949618 (+ 2368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:17.950410 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:17.950422 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:17.951155 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:17.952048 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:17.952079 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:17.953199 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:17.953208 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:17.955398 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:17.955418 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:17.956474 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:17.956483 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:17.956828 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:17.957734 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:17.957771 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:17.961016 (+ 3245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:17.965995 (+ 4979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:17.966005 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:17.966022 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:17.966607 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:17.967237 (+ 630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:17.967241 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:17.967244 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:17.967471 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:17.968291 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:17.968299 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:17.968761 (+ 462us) client_negotiation.cc:769] Sending connection context
0811 01:45:17.969033 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 01:45:17.969585 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":277,"threads_started":1}
I20260811 01:45:17.972613 32370 heartbeater.cc:347] Connected to a master server at 127.21.214.126:46459
I20260811 01:45:17.973006 32370 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:17.974118 32370 heartbeater.cc:511] Master 127.21.214.126:46459 requested a full tablet report, sending...
I20260811 01:45:17.976218 32003 ts_manager.cc:194] Registered new tserver with Master: 97bdd61c13e648c4bfeef76e16877d85 (127.21.214.66:33005)
I20260811 01:45:17.977241 32003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:43615
WARNING: no policy specified for kudu/127.21.214.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
I20260811 01:45:18.020234 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:46459
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:44933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:45:18.386232 32377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:18.386577 32377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:18.386749 32377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:18.394479 32377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:18.394662 32377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:18.394847 32377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:45:18.405208 32377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:44933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:46459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:18.406656 32377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:18.408169 32377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 01:45:18.416421 32218 heartbeater.cc:503] Master 127.21.214.126:46459 was elected leader, sending a full tablet report...
W20260811 01:45:18.422919 32383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:18.422984 32384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:18.422950 32387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:18.424459 32377 server_base.cc:1034] running on GCE node
I20260811 01:45:18.425362 32377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:18.426335 32377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:18.427541 32377 hybrid_clock.cc:648] HybridClock initialized: now 1786412718427467 us; error 64 us; skew 500 ppm
I20260811 01:45:18.427843 32377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:45:18 dist-test-slave-9gcw krb5kdc[31955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412718, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:45:18.431414 32377 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:45:18.433233 32377 webserver.cc:460] Webserver started at http://127.21.214.67:43649/ using document root <none> and password file <none>
I20260811 01:45:18.433995 32377 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:18.434128 32377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:18.434504 32377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:18.437834 32377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "d950dd75750641ae8ff1c40137c0457c"
format_stamp: "Formatted at 2026-08-11 01:45:18 on dist-test-slave-9gcw"
server_key: "b75706993c69b212f4f3401f57fb2a23"
server_key_iv: "394e4935575113632c3385414ee45ea4"
server_key_version: "encryptionkey@0"
I20260811 01:45:18.438819 32377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "d950dd75750641ae8ff1c40137c0457c"
format_stamp: "Formatted at 2026-08-11 01:45:18 on dist-test-slave-9gcw"
server_key: "b75706993c69b212f4f3401f57fb2a23"
server_key_iv: "394e4935575113632c3385414ee45ea4"
server_key_version: "encryptionkey@0"
I20260811 01:45:18.444487 32377 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 01:45:18.448573 32397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:18.450352 32377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:45:18.450548 32377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "d950dd75750641ae8ff1c40137c0457c"
format_stamp: "Formatted at 2026-08-11 01:45:18 on dist-test-slave-9gcw"
server_key: "b75706993c69b212f4f3401f57fb2a23"
server_key_iv: "394e4935575113632c3385414ee45ea4"
server_key_version: "encryptionkey@0"
I20260811 01:45:18.450775 32377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:18.466197 32377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:18.470984 32377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:18.471304 32377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:18.472436 32377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:45:18.473901 32377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:18.473977 32377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:18.474063 32377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:18.474112 32377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:18.514983 32377 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:33465
I20260811 01:45:18.515023 32526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:33465 every 8 connection(s)
I20260811 01:45:18.516358 32377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:45:18.520314 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 32377
I20260811 01:45:18.520498 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:45:18.521013 22361 external_mini_cluster.cc:1428] Setting key 9d7d2cb316439838ded96a357dd10009
Aug 11 01:45:18 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412718, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:45:18.538008 32530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:18.519234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:39431 (local address 127.21.214.126:46459)
0811 01:45:18.519729 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:18.519735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:18.520929 (+ 1194us) server_negotiation.cc:407] Connection header received
0811 01:45:18.523108 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:18.523115 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:18.523212 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:18.523308 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:18.525185 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:18.526006 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:18.527497 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:18.527895 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:18.527987 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:18.531704 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:18.531709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:18.531712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:18.531753 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:18.534496 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:18.535425 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:18.535432 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:18.535435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:18.535614 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:18.536158 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:18.536166 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:18.536169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:18.536563 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:18.536701 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:18.537510 (+ 809us) server_negotiation.cc:299] Negotiation successful
0811 01:45:18.537690 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":208,"threads_started":1}
I20260811 01:45:18.539029 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:18.519913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.67:39431)
0811 01:45:18.520741 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:18.520798 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:18.522824 (+ 2026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:18.523579 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:18.523591 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:18.524356 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:18.524994 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:18.525015 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:18.526137 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:18.526141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:18.527373 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:18.527386 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:18.528077 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:18.528081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:18.528274 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:18.528969 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:18.528992 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:18.531506 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:18.534687 (+ 3181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:18.534695 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:18.534712 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:18.535285 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:18.535748 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:18.535753 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:18.535756 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:18.536050 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:18.536707 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:18.536717 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:18.537091 (+ 374us) client_negotiation.cc:769] Sending connection context
0811 01:45:18.537413 (+ 322us) client_negotiation.cc:240] Negotiation successful
0811 01:45:18.537820 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":349,"threads_started":1}
I20260811 01:45:18.540524 32527 heartbeater.cc:347] Connected to a master server at 127.21.214.126:46459
I20260811 01:45:18.540917 32527 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:18.541730 32527 heartbeater.cc:511] Master 127.21.214.126:46459 requested a full tablet report, sending...
I20260811 01:45:18.543151 32003 ts_manager.cc:194] Registered new tserver with Master: d950dd75750641ae8ff1c40137c0457c (127.21.214.67:33465)
I20260811 01:45:18.543991 32003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:39431
I20260811 01:45:18.549185 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.65.keytab.
Entry for principal oryx/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.65.keytab.
WARNING: no policy specified for oryx/127.21.214.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.214.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.66.keytab.
Entry for principal oryx/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.66.keytab.
WARNING: no policy specified for oryx/127.21.214.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.214.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.67.keytab.
Entry for principal oryx/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.67.keytab.
I20260811 01:45:18.733229 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 32077
Aug 11 01:45:18 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:45:18.751868 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:18.734442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45246 (local address 127.21.214.65:45059)
0811 01:45:18.735292 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:18.735304 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:18.735696 (+ 392us) server_negotiation.cc:407] Connection header received
0811 01:45:18.735920 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:18.735928 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:18.736158 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:18.736309 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:18.737904 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:18.738993 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:18.740447 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:18.741070 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:18.741241 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:18.745907 (+ 4666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:18.745920 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:18.745937 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:18.745984 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:18.749129 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:18.749878 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:18.749886 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:18.749888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:18.750023 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:18.750348 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:18.750351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:18.750353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:18.750706 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:18.750947 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:18.751207 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 01:45:18.751416 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":405,"threads_started":1}
I20260811 01:45:18.753434 32211 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:18.839645 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 32077
I20260811 01:45:18.861374 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 32225
Aug 11 01:45:18 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.66@KRBTEST.COM
I20260811 01:45:18.878239 32543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:18.862387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52098 (local address 127.21.214.66:33005)
0811 01:45:18.862961 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:18.862968 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:18.862996 (+ 28us) server_negotiation.cc:407] Connection header received
0811 01:45:18.863061 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:18.863068 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:18.863301 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:18.863438 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:18.864751 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:18.866021 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:18.867311 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:18.867883 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:18.867995 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:18.872408 (+ 4413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:18.872421 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:18.872437 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:18.872473 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:18.875521 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:18.876187 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:18.876195 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:18.876197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:18.876320 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:18.876631 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:18.876634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:18.876636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:18.876999 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:18.877346 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:18.877689 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 01:45:18.877943 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":295,"threads_started":1}
I20260811 01:45:18.879678 32364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:18.969844 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 32225
I20260811 01:45:18.988103 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 32377
Aug 11 01:45:18 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.67@KRBTEST.COM
I20260811 01:45:19.005075 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:18.989263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48638 (local address 127.21.214.67:33465)
0811 01:45:18.989798 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:18.989806 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:18.989838 (+ 32us) server_negotiation.cc:407] Connection header received
0811 01:45:18.989901 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:18.989907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:18.990123 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:18.990304 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:18.991672 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:18.992650 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:18.994004 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:18.994570 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:18.994734 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:18.999203 (+ 4469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:18.999215 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:18.999230 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:18.999265 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:19.002215 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:19.002911 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:19.002918 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:19.002920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:19.003042 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:19.003434 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:19.003438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:19.003440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:19.003799 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:19.004108 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:19.004426 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 01:45:19.004708 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":253,"threads_started":1}
I20260811 01:45:19.006724 32521 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:19.092772 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 32377
I20260811 01:45:19.116087 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 31971
I20260811 01:45:19.117489 32036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:19.226408 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 31971
I20260811 01:45:19.252897 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:46459
--webserver_interface=127.21.214.126
--webserver_port=41467
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:44933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:46459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:45:19.602527 32549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:19.602851 32549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:19.602943 32549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:19.611793 32549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:45:19.611932 32549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:19.611990 32549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:19.612056 32549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:45:19.612106 32549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:45:19.623394 32549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:44933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:46459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:46459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=41467
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:19.624797 32549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:19.626180 32549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:19.639170 32554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:19.639144 32557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:19.639218 32555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:19.640185 32549 server_base.cc:1034] running on GCE node
I20260811 01:45:19.641185 32549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:19.642828 32549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:19.644099 32549 hybrid_clock.cc:648] HybridClock initialized: now 1786412719644032 us; error 57 us; skew 500 ppm
I20260811 01:45:19.644503 32549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:45:19 dist-test-slave-9gcw krb5kdc[31955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412719, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:45:19.648298 32549 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:45:19.650238 32549 webserver.cc:460] Webserver started at http://127.21.214.126:41467/ using document root <none> and password file <none>
I20260811 01:45:19.651002 32549 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:19.651171 32549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:19.657508 32549 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 01:45:19.660646 32564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:19.662432 32549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:45:19.662663 32549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "fe6bb40aef3f423cab96d5f64d6836b8"
format_stamp: "Formatted at 2026-08-11 01:45:16 on dist-test-slave-9gcw"
server_key: "1adbd3bc59e95a63c913951afffbe85a"
server_key_iv: "9fb1e5fac0f88c6a59b0f61a00e237e6"
server_key_version: "encryptionkey@0"
I20260811 01:45:19.664206 32549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:19.688225 32549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:19.694512 32549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:19.694929 32549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:19.720532 32549 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:46459
I20260811 01:45:19.720559 32621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:46459 every 8 connection(s)
I20260811 01:45:19.723151 32549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:45:19.732542 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 32549
I20260811 01:45:19.733131 32622 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:45:19.734341 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:45059
--local_ip_for_outbound_sockets=127.21.214.65
--tserver_master_addrs=127.21.214.126:46459
--webserver_port=36127
--webserver_interface=127.21.214.65
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:44933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.214.65
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:45:19.740227 32622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8: Bootstrap starting.
I20260811 01:45:19.766237 32622 log.cc:824] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8: Log is configured to *not* fsync() on all Append() calls
I20260811 01:45:19.778421 32622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:45:19.779278 32622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8: Bootstrap complete.
I20260811 01:45:19.791422 32622 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe6bb40aef3f423cab96d5f64d6836b8" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 46459 } }
I20260811 01:45:19.792665 32622 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe6bb40aef3f423cab96d5f64d6836b8, State: Initialized, Role: FOLLOWER
I20260811 01:45:19.793452 32622 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe6bb40aef3f423cab96d5f64d6836b8" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 46459 } }
I20260811 01:45:19.793831 32622 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:45:19.794000 32622 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:45:19.794314 32622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:45:19.800312 32622 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe6bb40aef3f423cab96d5f64d6836b8" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 46459 } }
I20260811 01:45:19.801124 32622 leader_election.cc:302] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe6bb40aef3f423cab96d5f64d6836b8; no voters:
I20260811 01:45:19.801924 32622 leader_election.cc:288] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 01:45:19.802135 32626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 01:45:19.803838 32626 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [term 2 LEADER]: Becoming Leader. State: Replica: fe6bb40aef3f423cab96d5f64d6836b8, State: Running, Role: LEADER
I20260811 01:45:19.804679 32626 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe6bb40aef3f423cab96d5f64d6836b8" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 46459 } }
I20260811 01:45:19.805338 32622 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:45:19.808667 32629 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe6bb40aef3f423cab96d5f64d6836b8. Latest consensus state: current_term: 2 leader_uuid: "fe6bb40aef3f423cab96d5f64d6836b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe6bb40aef3f423cab96d5f64d6836b8" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 46459 } } }
I20260811 01:45:19.809425 32629 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [sys.catalog]: This master's current role is: LEADER
I20260811 01:45:19.809587 32628 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fe6bb40aef3f423cab96d5f64d6836b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe6bb40aef3f423cab96d5f64d6836b8" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 46459 } } }
I20260811 01:45:19.810083 32628 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8 [sys.catalog]: This master's current role is: LEADER
I20260811 01:45:19.812968 32636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:45:19.828493 32636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:45:19.830791 32636 catalog_manager.cc:1259] Loaded cluster ID: 050b6d21ec164af5b03fceb26a8e3473
I20260811 01:45:19.831270 32636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:45:19.838831 32636 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:45:19.841253 32636 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fe6bb40aef3f423cab96d5f64d6836b8: Loaded TSK: 0
I20260811 01:45:19.843605 32636 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 01:45:20.191854 32624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:20.192301 32624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:20.192454 32624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:20.202890 32624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:20.203085 32624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:20.203263 32624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:45:20.216938 32624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:44933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.65.keytab
--principal=oryx/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:45059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=36127
--tserver_master_addrs=127.21.214.126:46459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:20.218884 32624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:20.220919 32624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:20.238615 32651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:20.238647 32652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:20.239224 32654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:20.240231 32624 server_base.cc:1034] running on GCE node
I20260811 01:45:20.241055 32624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:20.242627 32624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:20.244089 32624 hybrid_clock.cc:648] HybridClock initialized: now 1786412720243980 us; error 94 us; skew 500 ppm
I20260811 01:45:20.244643 32624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:45:20 dist-test-slave-9gcw krb5kdc[31955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412720, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:45:20.250734 32624 init.cc:377] Logged in from keytab as oryx/127.21.214.65@KRBTEST.COM (short username oryx)
I20260811 01:45:20.253063 32624 webserver.cc:460] Webserver started at http://127.21.214.65:36127/ using document root <none> and password file <none>
I20260811 01:45:20.254102 32624 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:20.254415 32624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:20.262022 32624 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.003s
I20260811 01:45:20.265502 32661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:20.267613 32624 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 01:45:20.267827 32624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "87cada0f4ddb4a879c15ef5027eac685"
format_stamp: "Formatted at 2026-08-11 01:45:17 on dist-test-slave-9gcw"
server_key: "8628e0ace4a6c433a7c489a71d7f3caf"
server_key_iv: "8a69daf7187ffa528ca49f84cd76a802"
server_key_version: "encryptionkey@0"
I20260811 01:45:20.269009 32624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:20.297489 32624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:20.302867 32624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:20.303171 32624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:20.304201 32624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:45:20.306123 32624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:20.306236 32624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:20.306360 32624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:20.306452 32624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:20.346820 32624 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:45059
I20260811 01:45:20.346910 318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:45059 every 8 connection(s)
I20260811 01:45:20.348349 32624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:45:20.354249 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 32624
I20260811 01:45:20.355126 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:33005
--local_ip_for_outbound_sockets=127.21.214.66
--tserver_master_addrs=127.21.214.126:46459
--webserver_port=44731
--webserver_interface=127.21.214.66
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:44933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.214.66
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 01:45:20 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:20 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:20.371132 323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:20.351862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:40807)
0811 01:45:20.352693 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:20.352755 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:20.354655 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:20.356748 (+ 2093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:20.356756 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:20.357353 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:20.357928 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:20.357948 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.360247 (+ 2299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.360251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:20.361875 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:20.361888 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.364123 (+ 2235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.364128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:20.364348 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:20.365089 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:20.365113 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:20.369656 (+ 4543us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":360,"threads_started":1}
W20260811 01:45:20.371143 322 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:20.351700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:40807 (local address 127.21.214.126:46459)
0811 01:45:20.352354 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:20.352366 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:20.352888 (+ 522us) server_negotiation.cc:407] Connection header received
0811 01:45:20.355432 (+ 2544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:20.355454 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:20.356082 (+ 628us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:20.356603 (+ 521us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:20.358141 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.360079 (+ 1938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.362054 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.363739 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.364118 (+ 379us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:20.370088 (+ 5970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.65:40807: BlockingRecv error: recv got EOF from 127.21.214.65:40807 (error 108)
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":257,"threads_started":1}
W20260811 01:45:20.371855 319 heartbeater.cc:644] Failed to heartbeat to 127.21.214.126:46459 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.214.126:46459: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Aug 11 01:45:20 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:20 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:20.383862 323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:20.372512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:50227)
0811 01:45:20.372718 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:20.372762 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:20.372995 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:20.373695 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:20.373703 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:20.374280 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:20.374540 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:20.374557 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.375692 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.375701 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:20.377441 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:20.377461 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.378225 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.378236 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:20.378475 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:20.379564 (+ 1089us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:20.379619 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:20.383445 (+ 3826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 01:45:20.384119 322 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:20.372679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:50227 (local address 127.21.214.126:46459)
0811 01:45:20.373358 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:20.373363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:20.373387 (+ 24us) server_negotiation.cc:407] Connection header received
0811 01:45:20.373441 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:20.373445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:20.373503 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:20.373648 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:20.374721 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.375533 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.377598 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.378049 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.378152 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:20.383725 (+ 5573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.65:50227: BlockingRecv error: recv got EOF from 127.21.214.65:50227 (error 108)
Metrics: {"server-negotiator.queue_time_us":411}
Aug 11 01:45:20 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:20 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:20.395028 323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:20.385039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:34335)
0811 01:45:20.385245 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:20.385277 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:20.385458 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:20.385925 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:20.385930 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:20.386330 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:20.386573 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:20.386584 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.387784 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.387790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:20.388892 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:20.388906 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.389654 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.389659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:20.389843 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:20.390498 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:20.390515 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:20.394708 (+ 4193us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 01:45:20.395452 319 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 01:45:20.395665 322 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:20.385096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:34335 (local address 127.21.214.126:46459)
0811 01:45:20.385273 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:20.385279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:20.385357 (+ 78us) server_negotiation.cc:407] Connection header received
0811 01:45:20.385634 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:20.385637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:20.385742 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:20.385900 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:20.386723 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.387505 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.389037 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.389509 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.389643 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:20.394959 (+ 5316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.65:34335: BlockingRecv error: recv got EOF from 127.21.214.65:34335 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 01:45:20.802707 324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:20.803023 324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:20.803189 324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:20.812911 324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:20.813189 324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:20.813609 324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:45:20.826978 324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:44933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.66.keytab
--principal=oryx/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:33005
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=44731
--tserver_master_addrs=127.21.214.126:46459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:20.828969 324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:20.831364 324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:20.849014 329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:20.849025 330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:20.849013 332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:20.850065 324 server_base.cc:1034] running on GCE node
I20260811 01:45:20.850790 324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:20.852137 324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:20.853361 324 hybrid_clock.cc:648] HybridClock initialized: now 1786412720853308 us; error 48 us; skew 500 ppm
I20260811 01:45:20.853828 324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:45:20 dist-test-slave-9gcw krb5kdc[31955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412720, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:45:20.857609 324 init.cc:377] Logged in from keytab as oryx/127.21.214.66@KRBTEST.COM (short username oryx)
I20260811 01:45:20.859411 324 webserver.cc:460] Webserver started at http://127.21.214.66:44731/ using document root <none> and password file <none>
I20260811 01:45:20.860345 324 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:20.860555 324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:20.867926 324 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 01:45:20.871307 339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:20.872862 324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 01:45:20.873118 324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "97bdd61c13e648c4bfeef76e16877d85"
format_stamp: "Formatted at 2026-08-11 01:45:17 on dist-test-slave-9gcw"
server_key: "d992f960fe82092354a55f64cd6e2356"
server_key_iv: "f04f0433f522aeff846b9515243111eb"
server_key_version: "encryptionkey@0"
I20260811 01:45:20.874580 324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:20.895195 324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:20.900770 324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:20.901273 324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:20.902431 324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:45:20.903995 324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:20.904086 324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:20.904245 324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:20.904333 324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:20.949532 324 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:33005
I20260811 01:45:20.949620 467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:33005 every 8 connection(s)
I20260811 01:45:20.951237 324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:45:20.955351 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 324
I20260811 01:45:20.956377 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:33465
--local_ip_for_outbound_sockets=127.21.214.67
--tserver_master_addrs=127.21.214.126:46459
--webserver_port=43649
--webserver_interface=127.21.214.67
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:44933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.214.67
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 01:45:20 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:20 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 01:45:20.972270 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:20.954283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:50459 (local address 127.21.214.126:46459)
0811 01:45:20.954805 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:20.954810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:20.956206 (+ 1396us) server_negotiation.cc:407] Connection header received
0811 01:45:20.958797 (+ 2591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:20.958805 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:20.958910 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:20.959043 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:20.960966 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.961896 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.963833 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.964323 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.964478 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:20.971919 (+ 7441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.66:50459: BlockingRecv error: recv got EOF from 127.21.214.66:50459 (error 108)
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":247,"threads_started":1}
I20260811 01:45:20.973228 472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:20.955015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:50459)
0811 01:45:20.955994 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:20.956044 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:20.958408 (+ 2364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:20.959329 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:20.959337 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:20.960108 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:20.960762 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:20.960782 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.962104 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.962108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:20.963606 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:20.963637 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.964605 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.964613 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:20.964953 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:20.966018 (+ 1065us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:20.966046 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:20.971681 (+ 5635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":408,"threads_started":1}
W20260811 01:45:20.974541 468 heartbeater.cc:644] Failed to heartbeat to 127.21.214.126:46459 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.214.126:46459: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Aug 11 01:45:20 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:20 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:20.985050 472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:20.975258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:60571)
0811 01:45:20.975452 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:20.975472 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:20.975598 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:20.976057 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:20.976061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:20.976468 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:20.976625 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:20.976633 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.977887 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.977892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:20.978879 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:20.978892 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.979926 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.979932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:20.980131 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:20.980893 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:20.980911 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:20.984678 (+ 3767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 01:45:20.985455 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:20.975434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:60571 (local address 127.21.214.126:46459)
0811 01:45:20.975607 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:20.975613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:20.975635 (+ 22us) server_negotiation.cc:407] Connection header received
0811 01:45:20.975807 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:20.975812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:20.975910 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:20.976022 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:20.976791 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.977741 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.979021 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.979769 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.979902 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:20.985093 (+ 5191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.66:60571: BlockingRecv error: recv got EOF from 127.21.214.66:60571 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 01:45:20 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:20 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:20.995643 472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:20.986285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:57327)
0811 01:45:20.986449 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:20.986472 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:20.986582 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:20.986975 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:20.986980 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:20.987338 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:20.987510 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:20.987518 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.988716 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.988722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:20.989895 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:20.989908 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.990674 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.990681 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:20.990876 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:20.991581 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:20.991598 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:20.995415 (+ 3817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 01:45:20.996058 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:20.986495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:57327 (local address 127.21.214.126:46459)
0811 01:45:20.986720 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:20.986724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:20.986739 (+ 15us) server_negotiation.cc:407] Connection header received
0811 01:45:20.986787 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:20.986791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:20.986846 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:20.986946 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:20.987713 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.988574 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.990045 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:20.990515 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:20.990643 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:20.995708 (+ 5065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.66:57327: BlockingRecv error: recv got EOF from 127.21.214.66:57327 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 01:45:20.996081 468 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 01:45:21.362702 473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:21.363013 473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:21.363108 473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:21.371203 473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:21.371332 473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:21.371447 473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:45:21.382234 473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:44933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/krb5kdc/oryx_127.21.214.67.keytab
--principal=oryx/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:33465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=43649
--tserver_master_addrs=127.21.214.126:46459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:21.383659 473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:21.384990 473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:21.400739 479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:21.401018 481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:21.401679 478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:21.402698 473 server_base.cc:1034] running on GCE node
I20260811 01:45:21.403420 473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:21.404752 473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Aug 11 01:45:21 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:21.405988 473 hybrid_clock.cc:648] HybridClock initialized: now 1786412721405927 us; error 52 us; skew 500 ppm
I20260811 01:45:21.406332 473 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:45:21 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:21.407857 482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:21.396238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:60187)
0811 01:45:21.396805 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:21.396833 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:21.397005 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:21.397571 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:21.397575 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:21.398052 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:21.398274 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:21.398284 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.400065 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.400069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:21.401835 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:21.401903 (+ 68us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.402978 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.402984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:21.403192 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:21.403933 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:21.403952 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:21.407495 (+ 3543us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":270,"threads_started":1}
W20260811 01:45:21.408020 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:21.396501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:60187 (local address 127.21.214.126:46459)
0811 01:45:21.396725 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:21.396732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:21.396933 (+ 201us) server_negotiation.cc:407] Connection header received
0811 01:45:21.397214 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:21.397222 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:21.397325 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:21.397448 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:21.398442 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.399897 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.402050 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.402754 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.402909 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:21.407739 (+ 4830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.65:60187: BlockingRecv error: recv got EOF from 127.21.214.65:60187 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 01:45:21 dist-test-slave-9gcw krb5kdc[31955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412721, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:45:21.410104 473 init.cc:377] Logged in from keytab as oryx/127.21.214.67@KRBTEST.COM (short username oryx)
I20260811 01:45:21.411738 473 webserver.cc:460] Webserver started at http://127.21.214.67:43649/ using document root <none> and password file <none>
I20260811 01:45:21.412487 473 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:21.412652 473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:21.418952 473 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 01:45:21.422158 489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:21.423939 473 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 01:45:21.424119 473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "d950dd75750641ae8ff1c40137c0457c"
format_stamp: "Formatted at 2026-08-11 01:45:18 on dist-test-slave-9gcw"
server_key: "b75706993c69b212f4f3401f57fb2a23"
server_key_iv: "394e4935575113632c3385414ee45ea4"
server_key_version: "encryptionkey@0"
I20260811 01:45:21.425340 473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:21.440661 473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:21.445536 473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:21.445834 473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:21.447157 473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:45:21.448729 473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:21.448856 473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:21.448976 473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:21.449061 473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:21.487913 473 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:33465
I20260811 01:45:21.488001 614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:33465 every 8 connection(s)
I20260811 01:45:21.489535 473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:45:21.494804 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 473
Aug 11 01:45:21 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:21 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 01:45:21.511194 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:21.492591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:57625 (local address 127.21.214.126:46459)
0811 01:45:21.492869 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:21.492875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:21.493978 (+ 1103us) server_negotiation.cc:407] Connection header received
0811 01:45:21.496747 (+ 2769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:21.496752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:21.496819 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:21.496941 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:21.499529 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.501056 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.503680 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.504324 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.504416 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:21.510884 (+ 6468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.67:57625: BlockingRecv error: recv got EOF from 127.21.214.67:57625 (error 108)
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":153}
I20260811 01:45:21.511816 619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:21.492925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.67:57625)
0811 01:45:21.493770 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:21.493850 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:21.495912 (+ 2062us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:21.497262 (+ 1350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:21.497272 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:21.498171 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:21.499296 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:21.499329 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.501355 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.501360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:21.503533 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:21.503554 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.504580 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.504587 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:21.504955 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:21.505790 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:21.505870 (+ 80us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:21.510626 (+ 4756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":350,"threads_started":1}
W20260811 01:45:21.512686 615 heartbeater.cc:644] Failed to heartbeat to 127.21.214.126:46459 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.214.126:46459: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
I20260811 01:45:21.515949 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:21.495751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40306 (local address 127.21.214.126:46459)
0811 01:45:21.497315 (+ 1564us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:21.497320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:21.497335 (+ 15us) server_negotiation.cc:407] Connection header received
0811 01:45:21.497378 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:21.497381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:21.497430 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:21.497517 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:21.499932 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.501352 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.503317 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.503894 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.504029 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:21.507748 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:21.507756 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:21.507779 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:21.507818 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:21.511598 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:21.512456 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:21.512466 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:21.512475 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:21.512731 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:21.513304 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:21.513311 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:21.513314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:21.514017 (+ 703us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:21.514309 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:21.514929 (+ 620us) server_negotiation.cc:299] Negotiation successful
0811 01:45:21.515451 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1461,"thread_start_us":1282,"threads_started":1}
Aug 11 01:45:21 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:21 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:21.522797 619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:21.513439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.67:51465)
0811 01:45:21.513639 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:21.513659 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:21.513771 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:21.514093 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:21.514100 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:21.514547 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:21.514885 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:21.514900 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.516401 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.516405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:21.517429 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:21.517442 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.518522 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.518532 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:21.518754 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:21.519405 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:21.519425 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:21.522594 (+ 3169us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 01:45:21.523068 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:21.513522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:51465 (local address 127.21.214.126:46459)
0811 01:45:21.513806 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:21.513811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:21.513829 (+ 18us) server_negotiation.cc:407] Connection header received
0811 01:45:21.513875 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:21.513880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:21.513947 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:21.514056 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:21.515066 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.516270 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.517583 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.518333 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.518453 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:21.522785 (+ 4332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.67:51465: BlockingRecv error: recv got EOF from 127.21.214.67:51465 (error 108)
Metrics: {"server-negotiator.queue_time_us":127}
Aug 11 01:45:21 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:21 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 01:45:21.536324 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:21.523636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:55099 (local address 127.21.214.126:46459)
0811 01:45:21.523878 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:21.523895 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:21.523915 (+ 20us) server_negotiation.cc:407] Connection header received
0811 01:45:21.524107 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:21.524112 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:21.524180 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:21.524274 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:21.525360 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.526476 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.528850 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.529710 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.529855 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:21.536023 (+ 6168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.67:55099: BlockingRecv error: recv got EOF from 127.21.214.67:55099 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 01:45:21.536885 619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:21.523529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.67:55099)
0811 01:45:21.523754 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:21.523775 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:21.523919 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:21.524333 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:21.524339 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:21.524838 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:21.525076 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:21.525092 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.526873 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.526877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:21.528617 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:21.528635 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.530057 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.530064 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:21.530306 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:21.531106 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:21.531134 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:21.536575 (+ 5441us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 01:45:21.537762 615 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 01:45:22 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:22 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:22.012274 622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:21.996842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:49315)
0811 01:45:21.997391 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:21.997409 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:21.997532 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:21.997862 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:21.997865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:21.998336 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:21.998531 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:21.998539 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.999587 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:21.999591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:22.000996 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:22.001018 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:22.002450 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:22.002456 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:22.002657 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:22.008598 (+ 5941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:22.008618 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:22.011958 (+ 3340us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":317,"threads_started":1}
W20260811 01:45:22.012568 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:21.996850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:49315 (local address 127.21.214.126:46459)
0811 01:45:21.997114 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:21.997126 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:21.997498 (+ 372us) server_negotiation.cc:407] Connection header received
0811 01:45:21.997640 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:21.997644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:21.997728 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:21.997824 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:21.998675 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:21.999450 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:22.001225 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:22.002232 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:22.002393 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:22.012284 (+ 9891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.66:49315: BlockingRecv error: recv got EOF from 127.21.214.66:49315 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 01:45:22 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:22 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:22.418702 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:22.408981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:43089)
0811 01:45:22.409535 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:22.409556 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:22.409689 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:22.410002 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:22.410005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:22.410467 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:22.410654 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:22.410663 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:22.411912 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:22.411920 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:22.413292 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:22.413312 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:22.414220 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:22.414226 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:22.414438 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:22.415063 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:22.415082 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:22.418301 (+ 3219us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":338,"threads_started":1}
W20260811 01:45:22.418889 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:22.409055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:43089 (local address 127.21.214.126:46459)
0811 01:45:22.409317 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:22.409324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:22.409638 (+ 314us) server_negotiation.cc:407] Connection header received
0811 01:45:22.409789 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:22.409793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:22.409874 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:22.409959 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:22.410792 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:22.411739 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:22.413542 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:22.414042 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:22.414209 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:22.418545 (+ 4336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.65:43089: BlockingRecv error: recv got EOF from 127.21.214.65:43089 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 01:45:22 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:22 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:22.548557 624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:22.538464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.67:37377)
0811 01:45:22.539101 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:22.539125 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:22.539319 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:22.539702 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:22.539707 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:22.540266 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:22.540494 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:22.540505 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:22.541809 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:22.541815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:22.543225 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:22.543241 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:22.544331 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:22.544336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:22.544485 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:22.545191 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:22.545211 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:22.548274 (+ 3063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":361,"threads_started":1}
W20260811 01:45:22.548722 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:22.538557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:37377 (local address 127.21.214.126:46459)
0811 01:45:22.538911 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:22.538920 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:22.539230 (+ 310us) server_negotiation.cc:407] Connection header received
0811 01:45:22.539461 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:22.539467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:22.539546 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:22.539664 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:22.540686 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:22.541659 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:22.543427 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:22.544022 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:22.544240 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:22.548509 (+ 4269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.67:37377: BlockingRecv error: recv got EOF from 127.21.214.67:37377 (error 108)
Metrics: {"server-negotiator.queue_time_us":161}
Aug 11 01:45:23 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:23 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:23.022755 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:23.013377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:38863)
0811 01:45:23.013973 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:23.013989 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:23.014234 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:23.014666 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:23.014671 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:23.015117 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:23.015296 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:23.015306 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:23.016364 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:23.016369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:23.017453 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:23.017467 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:23.018300 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:23.018307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:23.018473 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:23.019088 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:23.019106 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:23.022378 (+ 3272us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":385,"threads_started":1}
W20260811 01:45:23.022858 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:23.013409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:38863 (local address 127.21.214.126:46459)
0811 01:45:23.013612 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:23.013635 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:23.014082 (+ 447us) server_negotiation.cc:407] Connection header received
0811 01:45:23.014368 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:23.014373 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:23.014476 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:23.014646 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:23.015462 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:23.016216 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:23.017670 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:23.018114 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:23.018293 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:23.022609 (+ 4316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.66:38863: BlockingRecv error: recv got EOF from 127.21.214.66:38863 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 01:45:23 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:23 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:23.431027 626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:23.420088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:33807)
0811 01:45:23.420844 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:23.420875 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:23.421091 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:23.421551 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:23.421556 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:23.422029 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:23.422345 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:23.422357 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:23.423668 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:23.423675 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:23.424932 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:23.424965 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:23.426007 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:23.426017 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:23.426327 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:23.427233 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:23.427258 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:23.430730 (+ 3472us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":387,"threads_started":1}
W20260811 01:45:23.431573 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:23.420416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:33807 (local address 127.21.214.126:46459)
0811 01:45:23.420847 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:23.420856 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:23.420978 (+ 122us) server_negotiation.cc:407] Connection header received
0811 01:45:23.421300 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:23.421305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:23.421392 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:23.421513 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:23.422540 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:23.423441 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:23.425175 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:23.425809 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:23.425947 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:23.431082 (+ 5135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.65:33807: BlockingRecv error: recv got EOF from 127.21.214.65:33807 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
Aug 11 01:45:23 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:23 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:23.559371 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:23.549886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.67:42347)
0811 01:45:23.550440 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:23.550461 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:23.550641 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:23.551019 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:23.551027 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:23.551506 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:23.551776 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:23.551788 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:23.552961 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:23.552967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:23.554121 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:23.554135 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:23.555111 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:23.555118 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:23.555289 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:23.556056 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:23.556075 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:23.559111 (+ 3036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":303,"threads_started":1}
W20260811 01:45:23.559625 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:23.549973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:42347 (local address 127.21.214.126:46459)
0811 01:45:23.550258 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:23.550266 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:23.550554 (+ 288us) server_negotiation.cc:407] Connection header received
0811 01:45:23.550797 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:23.550801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:23.550866 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:23.550971 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:23.551960 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:23.552772 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:23.554319 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:23.554903 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:23.555037 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:23.559386 (+ 4349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.67:42347: BlockingRecv error: recv got EOF from 127.21.214.67:42347 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 01:45:24 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:24 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:24.033591 629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:24.023959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:47909)
0811 01:45:24.024597 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:24.024616 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:24.024752 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:24.025092 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:24.025096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:24.025559 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:24.025824 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:24.025835 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:24.026936 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:24.026951 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:24.027900 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:24.027914 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:24.028616 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:24.028620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:24.028916 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:24.029586 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:24.029605 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:24.033315 (+ 3710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":374,"threads_started":1}
W20260811 01:45:24.033859 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:24.024039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:47909 (local address 127.21.214.126:46459)
0811 01:45:24.024241 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:24.024246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:24.024705 (+ 459us) server_negotiation.cc:407] Connection header received
0811 01:45:24.024863 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:24.024867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:24.024948 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:24.025038 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:24.025990 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:24.026727 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:24.028034 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:24.028478 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:24.028577 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:24.033567 (+ 4990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.66:47909: BlockingRecv error: recv got EOF from 127.21.214.66:47909 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 01:45:24 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:24 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:24.442123 630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:24.432387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:42909)
0811 01:45:24.433046 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:24.433077 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:24.433274 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:24.433724 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:24.433730 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:24.434181 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:24.434370 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:24.434381 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:24.435507 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:24.435513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:24.436615 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:24.436630 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:24.437512 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:24.437518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:24.437740 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:24.438377 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:24.438396 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:24.441699 (+ 3303us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":399,"threads_started":1}
W20260811 01:45:24.442235 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:24.432495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:42909 (local address 127.21.214.126:46459)
0811 01:45:24.432770 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:24.432780 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:24.433210 (+ 430us) server_negotiation.cc:407] Connection header received
0811 01:45:24.433421 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:24.433428 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:24.433549 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:24.433670 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:24.434530 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:24.435317 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:24.436790 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:24.437346 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:24.437498 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:24.441925 (+ 4427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.65:42909: BlockingRecv error: recv got EOF from 127.21.214.65:42909 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 11 01:45:24 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:24 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:24.569839 631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:24.560499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.67:49135)
0811 01:45:24.561079 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:24.561128 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:24.561357 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:24.561705 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:24.561710 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:24.562130 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:24.562319 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:24.562328 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:24.563417 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:24.563424 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:24.564395 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:24.564408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:24.565180 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:24.565186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:24.565388 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:24.566126 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:24.566144 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:24.569368 (+ 3224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":341,"threads_started":1}
W20260811 01:45:24.569962 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:24.560532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:49135 (local address 127.21.214.126:46459)
0811 01:45:24.560833 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:24.560842 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:24.561275 (+ 433us) server_negotiation.cc:407] Connection header received
0811 01:45:24.561495 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:24.561500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:24.561570 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:24.561673 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:24.562496 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:24.563256 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:24.564548 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:24.564991 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:24.565145 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:24.569634 (+ 4489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.67:49135: BlockingRecv error: recv got EOF from 127.21.214.67:49135 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 11 01:45:25 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:25 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:25.044510 632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:25.034911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:53471)
0811 01:45:25.035514 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:25.035531 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:25.035713 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:25.036160 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:25.036164 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:25.036689 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:25.036850 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:25.036860 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:25.038148 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:25.038155 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:25.039306 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:25.039321 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:25.040102 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:25.040109 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:25.040299 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:25.040983 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:25.041002 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:25.044134 (+ 3132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":357,"threads_started":1}
W20260811 01:45:25.044648 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:25.034991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:53471 (local address 127.21.214.126:46459)
0811 01:45:25.035271 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:25.035278 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:25.035639 (+ 361us) server_negotiation.cc:407] Connection header received
0811 01:45:25.035921 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:25.035925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:25.035992 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:25.036103 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:25.037018 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:25.038003 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:25.039461 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:25.039950 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:25.040089 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:25.044390 (+ 4301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.66:53471: BlockingRecv error: recv got EOF from 127.21.214.66:53471 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 11 01:45:25 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:25 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:25.453025 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:25.443329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:50009)
0811 01:45:25.444017 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:25.444040 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:25.444205 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:25.444625 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:25.444630 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:25.445050 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:25.445290 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:25.445302 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:25.446869 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:25.446873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:25.448084 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:25.448098 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:25.449003 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:25.449008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:25.449217 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:25.449946 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:25.449964 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:25.452765 (+ 2801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":380,"threads_started":1}
W20260811 01:45:25.453411 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:25.443447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:50009 (local address 127.21.214.126:46459)
0811 01:45:25.443810 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:25.443817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:25.444134 (+ 317us) server_negotiation.cc:407] Connection header received
0811 01:45:25.444382 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:25.444387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:25.444476 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:25.444586 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:25.445445 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:25.446689 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:25.448240 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:25.448858 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:25.448990 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:25.452972 (+ 3982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.65:50009: BlockingRecv error: recv got EOF from 127.21.214.65:50009 (error 108)
Metrics: {"server-negotiator.queue_time_us":153}
Aug 11 01:45:25 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:25 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:25.580497 634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:25.571008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.67:53803)
0811 01:45:25.571595 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:25.571615 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:25.571796 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:25.572162 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:25.572167 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:25.572659 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:25.572837 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:25.572847 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:25.574227 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:25.574233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:25.575357 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:25.575371 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:25.576114 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:25.576119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:25.576307 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:25.576967 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:25.576986 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:25.580130 (+ 3144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":282,"threads_started":1}
W20260811 01:45:25.580646 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:25.571029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:53803 (local address 127.21.214.126:46459)
0811 01:45:25.571281 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:25.571289 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:25.571712 (+ 423us) server_negotiation.cc:407] Connection header received
0811 01:45:25.571936 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:25.571943 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:25.572017 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:25.572125 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:25.573031 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:25.574092 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:25.575512 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:25.575960 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:25.576101 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:25.580359 (+ 4258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.67:53803: BlockingRecv error: recv got EOF from 127.21.214.67:53803 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 01:45:26 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:26 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:26.054965 636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:26.045695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:60639)
0811 01:45:26.046348 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:26.046364 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:26.046528 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:26.046891 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:26.046896 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:26.047361 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:26.047530 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:26.047539 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:26.048709 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:26.048715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:26.049763 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:26.049776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:26.050543 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:26.050549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:26.050733 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:26.051334 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:26.051352 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:26.054680 (+ 3328us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":344,"threads_started":1}
W20260811 01:45:26.055171 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:26.045948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:60639 (local address 127.21.214.126:46459)
0811 01:45:26.046225 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:26.046232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:26.046450 (+ 218us) server_negotiation.cc:407] Connection header received
0811 01:45:26.046673 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:26.046679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:26.046754 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:26.046856 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:26.047678 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:26.048526 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:26.049909 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:26.050386 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:26.050530 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:26.054922 (+ 4392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.66:60639: BlockingRecv error: recv got EOF from 127.21.214.66:60639 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 11 01:45:26 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:26 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:26.463384 637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:26.454132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:42937)
0811 01:45:26.454732 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:26.454753 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:26.454928 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:26.455298 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:26.455302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:26.455741 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:26.455904 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:26.455913 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:26.456985 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:26.456990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:26.457992 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:26.458005 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:26.458904 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:26.458909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:26.459051 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:26.459834 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:26.459853 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:26.463047 (+ 3194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":347,"threads_started":1}
W20260811 01:45:26.463536 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:26.454266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:42937 (local address 127.21.214.126:46459)
0811 01:45:26.454501 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:26.454509 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:26.454847 (+ 338us) server_negotiation.cc:407] Connection header received
0811 01:45:26.455067 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:26.455071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:26.455149 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:26.455257 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:26.456055 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:26.456831 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:26.458144 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:26.458692 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:26.458853 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:26.463277 (+ 4424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.65:42937: BlockingRecv error: recv got EOF from 127.21.214.65:42937 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 01:45:26 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:26 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:26.590927 638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:26.581687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.67:33165)
0811 01:45:26.582237 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:26.582259 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:26.582385 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:26.582797 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:26.582800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:26.583246 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:26.583416 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:26.583426 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:26.584436 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:26.584439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:26.585535 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:26.585549 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:26.586336 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:26.586341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:26.586465 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:26.587207 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:26.587227 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:26.590613 (+ 3386us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":285,"threads_started":1}
W20260811 01:45:26.591058 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:26.581828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:33165 (local address 127.21.214.126:46459)
0811 01:45:26.582091 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:26.582097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:26.582351 (+ 254us) server_negotiation.cc:407] Connection header received
0811 01:45:26.582496 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:26.582500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:26.582565 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:26.582696 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:26.583551 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:26.584302 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:26.585704 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:26.586183 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:26.586287 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:26.590856 (+ 4569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.67:33165: BlockingRecv error: recv got EOF from 127.21.214.67:33165 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 11 01:45:27 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:27 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:27.069585 640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:27.056260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:50493)
0811 01:45:27.056862 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:27.056884 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:27.057071 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:27.057583 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:27.057588 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:27.058160 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:27.058332 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:27.058342 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:27.059457 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:27.059461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:27.060359 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:27.060371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:27.061067 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:27.061072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:27.061255 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:27.065661 (+ 4406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:27.065683 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:27.069221 (+ 3538us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":328,"threads_started":1}
W20260811 01:45:27.069840 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:27.056402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:50493 (local address 127.21.214.126:46459)
0811 01:45:27.056644 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:27.056651 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:27.056989 (+ 338us) server_negotiation.cc:407] Connection header received
0811 01:45:27.057271 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:27.057276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:27.057364 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:27.057483 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:27.058468 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:27.059306 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:27.060487 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:27.060917 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:27.061013 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:27.069525 (+ 8512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.66:50493: BlockingRecv error: recv got EOF from 127.21.214.66:50493 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 01:45:27 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:27 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:27.475549 641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:27.464596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:60047)
0811 01:45:27.465421 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:27.465444 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:27.465641 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:27.466091 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:27.466097 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:27.466581 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:27.466772 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:27.466783 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:27.467959 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:27.467965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:27.469444 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:27.469460 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:27.470561 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:27.470567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:27.470765 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:27.471473 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:27.471493 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:27.475238 (+ 3745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":352,"threads_started":1}
W20260811 01:45:27.475749 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:27.464649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:60047 (local address 127.21.214.126:46459)
0811 01:45:27.464949 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:27.464954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:27.465545 (+ 591us) server_negotiation.cc:407] Connection header received
0811 01:45:27.465846 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:27.465851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:27.465933 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:27.466054 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:27.466936 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:27.467787 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:27.469810 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:27.470377 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:27.470537 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:27.475493 (+ 4956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.65:60047: BlockingRecv error: recv got EOF from 127.21.214.65:60047 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
Aug 11 01:45:27 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:27 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:27.600302 642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:27.592028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.67:41397)
0811 01:45:27.592558 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:27.592580 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:27.592703 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:27.593015 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:27.593018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:27.593450 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:27.593615 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:27.593623 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:27.594713 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:27.594716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:27.595637 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:27.595649 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:27.596327 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:27.596331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:27.596456 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:27.597077 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:27.597094 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:27.600079 (+ 2985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":306,"threads_started":1}
W20260811 01:45:27.600582 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:27.592143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:41397 (local address 127.21.214.126:46459)
0811 01:45:27.592414 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:27.592422 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:27.592661 (+ 239us) server_negotiation.cc:407] Connection header received
0811 01:45:27.592814 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:27.592817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:27.592881 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:27.592967 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:27.593743 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:27.594582 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:27.595768 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:27.596182 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:27.596288 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:27.600324 (+ 4036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.67:41397: BlockingRecv error: recv got EOF from 127.21.214.67:41397 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 11 01:45:28 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:28 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:28.080273 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:28.070740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:43059)
0811 01:45:28.071353 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:28.071371 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:28.071513 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:28.071875 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:28.071879 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:28.072316 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:28.072499 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:28.072518 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:28.073944 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:28.073951 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:28.075046 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:28.075059 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:28.075803 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:28.075808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:28.075927 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:28.076626 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:28.076645 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:28.079952 (+ 3307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":266,"threads_started":1}
W20260811 01:45:28.080572 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:28.070773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:43059 (local address 127.21.214.126:46459)
0811 01:45:28.071063 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:28.071072 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:28.071460 (+ 388us) server_negotiation.cc:407] Connection header received
0811 01:45:28.071625 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:28.071629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:28.071713 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:28.071806 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:28.072668 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:28.073726 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:28.075211 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:28.075619 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:28.075740 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:28.080305 (+ 4565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.66:43059: BlockingRecv error: recv got EOF from 127.21.214.66:43059 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
Aug 11 01:45:28 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:28 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:28.486951 644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:28.476542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:39735)
0811 01:45:28.477182 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:28.477208 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:28.477355 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:28.477842 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:28.477846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:28.478378 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:28.478660 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:28.478675 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:28.480082 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:28.480087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:28.481103 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:28.481117 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:28.482157 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:28.482231 (+ 74us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:28.482397 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:28.483167 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:28.483185 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:28.486664 (+ 3479us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":316,"threads_started":1}
W20260811 01:45:28.487111 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:28.476819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:39735 (local address 127.21.214.126:46459)
0811 01:45:28.477034 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:28.477039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:28.477296 (+ 257us) server_negotiation.cc:407] Connection header received
0811 01:45:28.477478 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:28.477482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:28.477576 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:28.477716 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:28.478842 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:28.479912 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:28.481302 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:28.481816 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:28.481947 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:28.486843 (+ 4896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.65:39735: BlockingRecv error: recv got EOF from 127.21.214.65:39735 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 11 01:45:28 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:28 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:28.611351 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:28.601538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.67:34973)
0811 01:45:28.602296 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:28.602320 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:28.602475 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:28.602910 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:28.602913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:28.603401 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:28.603601 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:28.603615 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:28.604753 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:28.604757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:28.605903 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:28.605918 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:28.606814 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:28.606819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:28.606944 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:28.607533 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:28.607550 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:28.611086 (+ 3536us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":331,"threads_started":1}
W20260811 01:45:28.611557 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:28.601693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:34973 (local address 127.21.214.126:46459)
0811 01:45:28.601862 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:28.601867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:28.602404 (+ 537us) server_negotiation.cc:407] Connection header received
0811 01:45:28.602653 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:28.602660 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:28.602743 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:28.602846 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:28.603780 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:28.604591 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:28.606049 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:28.606650 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:28.606753 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:28.611300 (+ 4547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.67:34973: BlockingRecv error: recv got EOF from 127.21.214.67:34973 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 01:45:29 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:29 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:29.090479 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:29.081294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:37461)
0811 01:45:29.081885 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:29.081901 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:29.082025 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:29.082409 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:29.082412 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:29.082824 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:29.082981 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:29.082990 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:29.084426 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:29.084430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:29.085431 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:29.085444 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:29.086255 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:29.086259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:29.086386 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:29.087089 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:29.087106 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:29.090228 (+ 3122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":321,"threads_started":1}
W20260811 01:45:29.090790 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:29.081418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:37461 (local address 127.21.214.126:46459)
0811 01:45:29.081609 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:29.081613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:29.082016 (+ 403us) server_negotiation.cc:407] Connection header received
0811 01:45:29.082173 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:29.082178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:29.082252 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:29.082359 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:29.083141 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:29.084276 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:29.085569 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:29.086083 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:29.086205 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:29.090506 (+ 4301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.66:37461: BlockingRecv error: recv got EOF from 127.21.214.66:37461 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 01:45:29 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:29 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:29.497777 647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:29.488113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:55027)
0811 01:45:29.488711 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:29.488743 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:29.488948 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:29.489337 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:29.489342 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:29.489912 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:29.490123 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:29.490135 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:29.491236 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:29.491242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:29.492397 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:29.492411 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:29.493122 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:29.493127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:29.493327 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:29.493946 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:29.493964 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:29.497518 (+ 3554us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":305,"threads_started":1}
W20260811 01:45:29.497985 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:29.488210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:55027 (local address 127.21.214.126:46459)
0811 01:45:29.488491 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:29.488498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:29.488880 (+ 382us) server_negotiation.cc:407] Connection header received
0811 01:45:29.489073 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:29.489077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:29.489181 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:29.489269 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:29.490307 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:29.491071 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:29.492531 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:29.492968 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:29.493070 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:29.497758 (+ 4688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.65:55027: BlockingRecv error: recv got EOF from 127.21.214.65:55027 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 11 01:45:29 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:29 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:29.621703 648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:29.612363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.67:37595)
0811 01:45:29.613020 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:29.613041 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:29.613230 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:29.613541 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:29.613545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:29.614057 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:29.614222 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:29.614231 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:29.615314 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:29.615318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:29.616278 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:29.616291 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:29.617068 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:29.617077 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:29.617250 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:29.618002 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:29.618022 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:29.621350 (+ 3328us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":370,"threads_started":1}
W20260811 01:45:29.621851 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:29.612484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:37595 (local address 127.21.214.126:46459)
0811 01:45:29.612665 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:29.612669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:29.613131 (+ 462us) server_negotiation.cc:407] Connection header received
0811 01:45:29.613340 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:29.613343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:29.613408 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:29.613495 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:29.614371 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:29.615143 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:29.616414 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:29.616853 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:29.616952 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:29.621586 (+ 4634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.67:37595: BlockingRecv error: recv got EOF from 127.21.214.67:37595 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 01:45:30 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:30 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:30.100464 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:30.091578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:41787)
0811 01:45:30.092246 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:30.092263 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:30.092383 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:30.092836 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:30.092839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:30.093275 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:30.093450 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:30.093459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:30.094714 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:30.094718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:30.095688 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:30.095701 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:30.096463 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:30.096467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:30.096590 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:30.097296 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:30.097314 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:30.100202 (+ 2888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":395,"threads_started":1}
W20260811 01:45:30.100800 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:30.091622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:41787 (local address 127.21.214.126:46459)
0811 01:45:30.091818 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:30.091826 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:30.092355 (+ 529us) server_negotiation.cc:407] Connection header received
0811 01:45:30.092538 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:30.092542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:30.092664 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:30.092764 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:30.093591 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:30.094554 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:30.095852 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:30.096320 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:30.096430 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:30.100444 (+ 4014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.66:41787: BlockingRecv error: recv got EOF from 127.21.214.66:41787 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 01:45:30 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:30 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:30.508602 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:30.498857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:50209)
0811 01:45:30.499412 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:30.499434 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:30.499550 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:30.499890 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:30.499893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:30.500319 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:30.500482 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:30.500491 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:30.501635 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:30.501639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:30.502598 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:30.502610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:30.503374 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:30.503380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:30.503522 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:30.504458 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:30.504483 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:30.508371 (+ 3888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":313,"threads_started":1}
W20260811 01:45:30.508930 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:30.499079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:50209 (local address 127.21.214.126:46459)
0811 01:45:30.499262 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:30.499266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:30.499515 (+ 249us) server_negotiation.cc:407] Connection header received
0811 01:45:30.499650 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:30.499659 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:30.499744 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:30.499840 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:30.500625 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:30.501479 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:30.502735 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:30.503185 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:30.503287 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:30.508572 (+ 5285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.65:50209: BlockingRecv error: recv got EOF from 127.21.214.65:50209 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 01:45:30 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:30 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:30.632661 651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:30.622861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.67:54011)
0811 01:45:30.623496 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:30.623520 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:30.623656 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:30.624085 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:30.624090 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:30.624495 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:30.624686 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:30.624695 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:30.625846 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:30.625851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:30.626834 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:30.626848 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:30.627617 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:30.627622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:30.627822 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:30.628512 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:30.628532 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:30.632346 (+ 3814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":338,"threads_started":1}
W20260811 01:45:30.632804 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:30.622987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:54011 (local address 127.21.214.126:46459)
0811 01:45:30.623211 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:30.623215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:30.623641 (+ 426us) server_negotiation.cc:407] Connection header received
0811 01:45:30.623808 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:30.623813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:30.623880 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:30.623975 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:30.624850 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:30.625640 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:30.626998 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:30.627457 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:30.627562 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:30.632567 (+ 5005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.67:54011: BlockingRecv error: recv got EOF from 127.21.214.67:54011 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 01:45:31 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:31 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:31.110453 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:31.101491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:44867)
0811 01:45:31.102016 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:31.102034 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:31.102160 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:31.102475 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:31.102479 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:31.102881 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:31.103040 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:31.103048 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:31.104119 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:31.104123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:31.105096 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:31.105109 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:31.105948 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:31.105954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:31.106134 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:31.106820 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:31.106838 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:31.110099 (+ 3261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":297,"threads_started":1}
W20260811 01:45:31.110628 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:31.101614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:44867 (local address 127.21.214.126:46459)
0811 01:45:31.101956 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:31.101964 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:31.102117 (+ 153us) server_negotiation.cc:407] Connection header received
0811 01:45:31.102266 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:31.102270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:31.102331 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:31.102414 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:31.103177 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:31.103982 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:31.105263 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:31.105757 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:31.105920 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:31.110338 (+ 4418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.66:44867: BlockingRecv error: recv got EOF from 127.21.214.66:44867 (error 108)
Metrics: {"server-negotiator.queue_time_us":208}
Aug 11 01:45:31 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:31 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:31.522527 653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:31.509678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:38571)
0811 01:45:31.510233 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:31.510256 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:31.510405 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:31.510769 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:31.510773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:31.511255 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:31.511464 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:31.511474 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:31.512703 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:31.512708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:31.513817 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:31.513832 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:31.514614 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:31.514618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:31.514742 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:31.518659 (+ 3917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:31.518678 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:31.522263 (+ 3585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":304,"threads_started":1}
W20260811 01:45:31.522708 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:31.509885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:38571 (local address 127.21.214.126:46459)
0811 01:45:31.510222 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:31.510229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:31.510363 (+ 134us) server_negotiation.cc:407] Connection header received
0811 01:45:31.510512 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:31.510516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:31.510584 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:31.510688 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:31.511641 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:31.512562 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:31.513964 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:31.514471 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:31.514582 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:31.522480 (+ 7898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.65:38571: BlockingRecv error: recv got EOF from 127.21.214.65:38571 (error 108)
Metrics: {"server-negotiator.queue_time_us":147}
Aug 11 01:45:31 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:31 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:31.643404 654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:31.633736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.67:32903)
0811 01:45:31.634343 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:31.634372 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:31.634495 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:31.634862 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:31.634866 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:31.635293 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:31.635486 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:31.635495 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:31.636661 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:31.636666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:31.637792 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:31.637807 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:31.638813 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:31.638820 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:31.639004 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:31.639655 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:31.639675 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:31.643056 (+ 3381us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":354,"threads_started":1}
W20260811 01:45:31.643563 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:31.633789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:32903 (local address 127.21.214.126:46459)
0811 01:45:31.634105 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:31.634114 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:31.634462 (+ 348us) server_negotiation.cc:407] Connection header received
0811 01:45:31.634594 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:31.634598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:31.634682 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:31.634804 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:31.635688 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:31.636490 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:31.637966 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:31.638578 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:31.638732 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:31.643283 (+ 4551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.67:32903: BlockingRecv error: recv got EOF from 127.21.214.67:32903 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 01:45:32 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:32 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:32.124703 655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:32.111490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:59667)
0811 01:45:32.112092 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:32.112109 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:32.112257 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:32.112580 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:32.112583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:32.113199 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:32.113413 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:32.113438 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:32.114459 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:32.114463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:32.115370 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:32.115382 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:32.116048 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:32.116052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:32.116185 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:32.120988 (+ 4803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:32.121011 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:32.124439 (+ 3428us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":353,"threads_started":1}
W20260811 01:45:32.125088 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:32.111648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:59667 (local address 127.21.214.126:46459)
0811 01:45:32.111904 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:32.111911 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:32.112218 (+ 307us) server_negotiation.cc:407] Connection header received
0811 01:45:32.112372 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:32.112375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:32.112440 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:32.112524 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:32.113578 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:32.114336 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:32.115497 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:32.115911 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:32.116005 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:32.124736 (+ 8731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.66:59667: BlockingRecv error: recv got EOF from 127.21.214.66:59667 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 11 01:45:32 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:32 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:32.532475 656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:32.523575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:34861)
0811 01:45:32.524069 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:32.524090 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:32.524210 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:32.524545 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:32.524548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:32.524960 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:32.525118 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:32.525127 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:32.526296 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:32.526300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:32.527260 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:32.527272 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:32.527947 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:32.527951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:32.528085 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:32.528798 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:32.528818 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:32.532123 (+ 3305us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":294,"threads_started":1}
W20260811 01:45:32.532696 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:32.523650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:34861 (local address 127.21.214.126:46459)
0811 01:45:32.523907 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:32.523913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:32.524200 (+ 287us) server_negotiation.cc:407] Connection header received
0811 01:45:32.524318 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:32.524322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:32.524387 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:32.524472 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:32.525327 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:32.526161 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:32.527407 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:32.527815 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:32.527915 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:32.532404 (+ 4489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.65:34861: BlockingRecv error: recv got EOF from 127.21.214.65:34861 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 01:45:32 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:32 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:32.653189 657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:32.644647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.67:52075)
0811 01:45:32.645285 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:32.645310 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:32.645435 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:32.645813 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:32.645817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:32.646304 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:32.646489 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:32.646497 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:32.647510 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:32.647513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:32.648404 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:32.648415 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:32.649127 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:32.649131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:32.649313 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:32.649976 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:32.649994 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:32.652894 (+ 2900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":361,"threads_started":1}
W20260811 01:45:32.653441 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:32.644756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:52075 (local address 127.21.214.126:46459)
0811 01:45:32.645019 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:32.645026 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:32.645402 (+ 376us) server_negotiation.cc:407] Connection header received
0811 01:45:32.645556 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:32.645561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:32.645638 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:32.645723 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:32.646635 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:32.647379 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:32.648547 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:32.648990 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:32.649092 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:32.653106 (+ 4014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.67:52075: BlockingRecv error: recv got EOF from 127.21.214.67:52075 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 11 01:45:33 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:33 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:33.136067 658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:33.125713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:57795)
0811 01:45:33.126296 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:33.126320 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:33.126508 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:33.126908 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:33.126912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:33.127420 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:33.127583 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:33.127592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:33.128789 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:33.128795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:33.130250 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:33.130264 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:33.131105 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:33.131112 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:33.131329 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:33.131994 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:33.132014 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:33.135711 (+ 3697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":309,"threads_started":1}
W20260811 01:45:33.136205 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:33.126032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:57795 (local address 127.21.214.126:46459)
0811 01:45:33.126299 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:33.126308 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:33.126433 (+ 125us) server_negotiation.cc:407] Connection header received
0811 01:45:33.126646 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:33.126653 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:33.126748 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:33.126876 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:33.127753 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:33.128630 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:33.130439 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:33.130922 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:33.131076 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:33.135938 (+ 4862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.66:57795: BlockingRecv error: recv got EOF from 127.21.214.66:57795 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 01:45:33 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:33 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:33.558182 659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:33.533833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:57153)
0811 01:45:33.548940 (+ 15107us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:33.548984 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:33.549215 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:33.549619 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:33.549625 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:33.550088 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:33.550304 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:33.550314 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:33.551357 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:33.551362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:33.552538 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:33.552556 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:33.553726 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:33.553735 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:33.553950 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:33.554688 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:33.554709 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:33.557814 (+ 3105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":14956,"thread_start_us":15332,"threads_started":1}
W20260811 01:45:33.558302 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:33.533911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:57153 (local address 127.21.214.126:46459)
0811 01:45:33.534161 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:33.534172 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:33.549120 (+ 14948us) server_negotiation.cc:407] Connection header received
0811 01:45:33.549393 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:33.549397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:33.549470 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:33.549566 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:33.550454 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:33.551223 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:33.552744 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:33.553413 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:33.553512 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:33.558060 (+ 4548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.65:57153: BlockingRecv error: recv got EOF from 127.21.214.65:57153 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 01:45:33 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:33 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:33.663727 660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:33.654275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.67:45609)
0811 01:45:33.654946 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:33.654969 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:33.655088 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:33.655406 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:33.655409 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:33.655928 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:33.656168 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:33.656182 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:33.657465 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:33.657470 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:33.658569 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:33.658594 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:33.659335 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:33.659340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:33.659487 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:33.660191 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:33.660211 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:33.663366 (+ 3155us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":434,"threads_started":1}
W20260811 01:45:33.663914 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:33.654326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:45609 (local address 127.21.214.126:46459)
0811 01:45:33.654528 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:33.654536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:33.655049 (+ 513us) server_negotiation.cc:407] Connection header received
0811 01:45:33.655193 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:33.655196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:33.655258 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:33.655353 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:33.656348 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:33.657310 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:33.658735 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:33.659185 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:33.659282 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:33.663589 (+ 4307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.67:45609: BlockingRecv error: recv got EOF from 127.21.214.67:45609 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 01:45:34 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:34 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:34.147146 662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:34.137391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:52323)
0811 01:45:34.138078 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:34.138095 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:34.138296 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:34.138671 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:34.138676 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:34.139126 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:34.139291 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:34.139299 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:34.140482 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:34.140488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:34.141499 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:34.141511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:34.142456 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:34.142462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:34.142647 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:34.143502 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:34.143525 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:34.146886 (+ 3361us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":385,"threads_started":1}
W20260811 01:45:34.147360 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:34.137700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:52323 (local address 127.21.214.126:46459)
0811 01:45:34.137975 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:34.137982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:34.138217 (+ 235us) server_negotiation.cc:407] Connection header received
0811 01:45:34.138434 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:34.138440 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:34.138512 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:34.138646 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:34.139446 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:34.140325 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:34.141660 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:34.142232 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:34.142398 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:34.147104 (+ 4706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.66:52323: BlockingRecv error: recv got EOF from 127.21.214.66:52323 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
Aug 11 01:45:34 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:34 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:34.568291 663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:34.559257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:35771)
0811 01:45:34.559805 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:34.559829 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:34.559950 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:34.560280 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:34.560283 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:34.560757 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:34.560941 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:34.560951 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:34.562116 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:34.562121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:34.563090 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:34.563104 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:34.563858 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:34.563863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:34.563992 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:34.564624 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:34.564642 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:34.567876 (+ 3234us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":294,"threads_started":1}
W20260811 01:45:34.568383 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:34.559521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:35771 (local address 127.21.214.126:46459)
0811 01:45:34.559742 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:34.559747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:34.559944 (+ 197us) server_negotiation.cc:407] Connection header received
0811 01:45:34.560059 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:34.560063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:34.560130 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:34.560211 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:34.561106 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:34.561943 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:34.563242 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:34.563702 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:34.563803 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:34.568111 (+ 4308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.65:35771: BlockingRecv error: recv got EOF from 127.21.214.65:35771 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 01:45:34 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:34 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:34.674750 664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:34.664878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.67:59837)
0811 01:45:34.665676 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:34.665697 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:34.665831 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:34.666149 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:34.666153 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:34.666682 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:34.666908 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:34.666918 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:34.668069 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:34.668073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:34.669010 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:34.669021 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:34.669968 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:34.669972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:34.670150 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:34.670943 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:34.670962 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:34.674372 (+ 3410us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":514,"threads_started":1}
W20260811 01:45:34.674821 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:34.665004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:59837 (local address 127.21.214.126:46459)
0811 01:45:34.665384 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:34.665395 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:34.665812 (+ 417us) server_negotiation.cc:407] Connection header received
0811 01:45:34.665942 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:34.665946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:34.666013 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:34.666108 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:34.667100 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:34.667930 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:34.669229 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:34.669741 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:34.669894 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:34.674585 (+ 4691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.67:59837: BlockingRecv error: recv got EOF from 127.21.214.67:59837 (error 108)
Metrics: {"server-negotiator.queue_time_us":165}
Aug 11 01:45:35 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:35 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:35.157507 665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:35.148476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:41185)
0811 01:45:35.149111 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:35.149133 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:35.149338 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:35.149700 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:35.149705 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:35.150147 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:35.150336 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:35.150345 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:35.151512 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:35.151517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:35.152436 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:35.152447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:35.153208 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:35.153215 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:35.153394 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:35.154044 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:35.154063 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:35.157062 (+ 2999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":415,"threads_started":1}
W20260811 01:45:35.157630 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:35.148589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:41185 (local address 127.21.214.126:46459)
0811 01:45:35.148853 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:35.148860 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:35.149273 (+ 413us) server_negotiation.cc:407] Connection header received
0811 01:45:35.149488 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:35.149492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:35.149556 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:35.149673 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:35.150490 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:35.151366 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:35.152585 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:35.153013 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:35.153149 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:35.157327 (+ 4178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.66:41185: BlockingRecv error: recv got EOF from 127.21.214.66:41185 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 11 01:45:35 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:35 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.65@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:35.579600 666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:35.569875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.65:35713)
0811 01:45:35.570533 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:35.570554 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:35.570703 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:35.571224 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:35.571228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:35.571803 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:35.572091 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:35.572104 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:35.573537 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:35.573548 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:35.574559 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:35.574572 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:35.575384 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:35.575388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:35.575508 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:35.576173 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:35.576191 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:35.579263 (+ 3072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":357,"threads_started":1}
W20260811 01:45:35.579838 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:35.569944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:35713 (local address 127.21.214.126:46459)
0811 01:45:35.570278 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:35.570285 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:35.570757 (+ 472us) server_negotiation.cc:407] Connection header received
0811 01:45:35.570901 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:35.570907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:35.571021 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:35.571152 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:35.572276 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:35.573384 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:35.574768 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:35.575239 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:35.575359 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:35.579479 (+ 4120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.65:35713: BlockingRecv error: recv got EOF from 127.21.214.65:35713 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 01:45:35 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:35 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.67@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:35.686029 667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:35.675967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.67:41943)
0811 01:45:35.676441 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:35.676463 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:35.676585 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:35.676995 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:35.676998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:35.677438 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:35.677676 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:35.677687 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:35.679164 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:35.679168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:35.680330 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:35.680345 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:35.681046 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:35.681051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:35.681312 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:35.681962 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:35.681982 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:35.685593 (+ 3611us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":263,"threads_started":1}
W20260811 01:45:35.686116 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:35.676039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:41943 (local address 127.21.214.126:46459)
0811 01:45:35.676303 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:35.676313 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:35.676561 (+ 248us) server_negotiation.cc:407] Connection header received
0811 01:45:35.676790 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:35.676794 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:35.676859 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:35.676946 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:35.677856 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:35.679006 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:35.680468 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:35.680880 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:35.680982 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:35.685858 (+ 4876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.67:41943: BlockingRecv error: recv got EOF from 127.21.214.67:41943 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 01:45:36 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 01:45:36 dist-test-slave-9gcw krb5kdc[31955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.214.66@KRBTEST.COM for oryx/127.21.214.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 01:45:36.167913 668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:36.159010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:46459 (local address 127.21.214.66:56095)
0811 01:45:36.159800 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:36.159817 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:36.159929 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:36.160229 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:36.160232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:36.160655 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:36.160822 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:36.160831 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:36.162148 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:36.162151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:36.163067 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:36.163079 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:36.163872 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:36.163876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:36.163995 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:36.164584 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:36.164601 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:36.167658 (+ 3057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.214.126:46459: Server oryx/127.21.214.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":519,"threads_started":1}
W20260811 01:45:36.168078 471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:36.159146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:56095 (local address 127.21.214.126:46459)
0811 01:45:36.159559 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:36.159565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:36.159902 (+ 337us) server_negotiation.cc:407] Connection header received
0811 01:45:36.160031 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:36.160034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:36.160100 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:36.160188 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:36.160992 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:36.162009 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:36.163220 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:36.163736 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:36.163845 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:36.167876 (+ 4031us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.214.66:56095: BlockingRecv error: recv got EOF from 127.21.214.66:56095 (error 108)
Metrics: {"server-negotiator.queue_time_us":269}
I20260811 01:45:36.496488 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 32624
I20260811 01:45:36.509768 670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:36.497824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56574 (local address 127.21.214.65:45059)
0811 01:45:36.498489 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:36.498496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:36.498525 (+ 29us) server_negotiation.cc:407] Connection header received
0811 01:45:36.498590 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:36.498597 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:36.498833 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:36.498999 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:36.500479 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:36.501636 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:36.502937 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:36.503566 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:36.503700 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:36.506424 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:36.506433 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:36.506448 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:36.506481 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:36.509232 (+ 2751us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.214.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 01:45:36.509525 (+ 293us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56574: Request ticket server kudu/127.21.214.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":319,"threads_started":1}
W20260811 01:45:36.509970 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 32624: Not authorized: Client connection negotiation failed: client connection to 127.21.214.65:45059: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.214.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 01:45:36.510202 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 32624
I20260811 01:45:36.532943 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 324
I20260811 01:45:36.546123 671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:36.534191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54868 (local address 127.21.214.66:33005)
0811 01:45:36.534915 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:36.534922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:36.534958 (+ 36us) server_negotiation.cc:407] Connection header received
0811 01:45:36.535028 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:36.535035 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:36.535282 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:36.535477 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:36.536805 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:36.538130 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:36.539336 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:36.539936 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:36.540054 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:36.542683 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:36.542693 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:36.542709 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:36.542744 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:36.545522 (+ 2778us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.214.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 01:45:36.545831 (+ 309us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54868: Request ticket server kudu/127.21.214.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":355,"threads_started":1}
W20260811 01:45:36.546276 671 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54868: Request ticket server kudu/127.21.214.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 01:45:36.546279 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 324: Not authorized: Client connection negotiation failed: client connection to 127.21.214.66:33005: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.214.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 01:45:36.546439 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 324
I20260811 01:45:36.568734 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 473
I20260811 01:45:36.581933 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:36.569909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46102 (local address 127.21.214.67:33465)
0811 01:45:36.570552 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:36.570560 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:36.570587 (+ 27us) server_negotiation.cc:407] Connection header received
0811 01:45:36.570650 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:36.570656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:36.570965 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:36.571122 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:36.572457 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:36.573594 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:36.574739 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:36.575312 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:36.575431 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:36.578568 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:36.578577 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:36.578592 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:36.578626 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:36.581375 (+ 2749us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.214.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 01:45:36.581652 (+ 277us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46102: Request ticket server kudu/127.21.214.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":368,"threads_started":1}
W20260811 01:45:36.582047 672 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46102: Request ticket server kudu/127.21.214.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 01:45:36.582119 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 473: Not authorized: Client connection negotiation failed: client connection to 127.21.214.67:33465: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.214.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 01:45:36.582346 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 473
I20260811 01:45:36.604621 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 32549
I20260811 01:45:36.606391 32617 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:36.727439 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 32549
2026-08-11T01:45:36Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22055 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T01:45:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:45:36Z Disabled control of system clock
I20260811 01:45:36.827400 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:43319
--webserver_interface=127.21.214.126
--webserver_port=0
--builtin_ntp_servers=127.21.214.84:36487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:43319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 01:45:37.203414 677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:37.203778 677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:37.203867 677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:37.211745 677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:45:37.211900 677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:37.211947 677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:37.212013 677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:45:37.212066 677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:45:37.222095 677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:36487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:43319
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:43319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:37.223549 677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:37.225688 677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:37.239562 686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:37.239794 683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:37.240300 682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:37.245905 677 server_base.cc:1034] running on GCE node
I20260811 01:45:37.247051 677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:37.248929 677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:37.250181 677 hybrid_clock.cc:648] HybridClock initialized: now 1786412737250120 us; error 60 us; skew 500 ppm
I20260811 01:45:37.250634 677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:45:37.258046 677 webserver.cc:460] Webserver started at http://127.21.214.126:38037/ using document root <none> and password file <none>
I20260811 01:45:37.258970 677 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:37.259127 677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:37.259532 677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:37.263280 677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "118d948343064be4800d37d062b1e82e"
format_stamp: "Formatted at 2026-08-11 01:45:37 on dist-test-slave-9gcw"
server_key: "d6eac24d496dd5a86eb558313ef49347"
server_key_iv: "564b644a1232bc208b8f304f2d7482ad"
server_key_version: "encryptionkey@0"
I20260811 01:45:37.264302 677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "118d948343064be4800d37d062b1e82e"
format_stamp: "Formatted at 2026-08-11 01:45:37 on dist-test-slave-9gcw"
server_key: "d6eac24d496dd5a86eb558313ef49347"
server_key_iv: "564b644a1232bc208b8f304f2d7482ad"
server_key_version: "encryptionkey@0"
I20260811 01:45:37.270759 677 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 01:45:37.274780 692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:37.276312 677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:45:37.276523 677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "118d948343064be4800d37d062b1e82e"
format_stamp: "Formatted at 2026-08-11 01:45:37 on dist-test-slave-9gcw"
server_key: "d6eac24d496dd5a86eb558313ef49347"
server_key_iv: "564b644a1232bc208b8f304f2d7482ad"
server_key_version: "encryptionkey@0"
I20260811 01:45:37.276844 677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:37.336488 677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:37.337672 677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:37.338007 677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:37.358144 677 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:43319
I20260811 01:45:37.358165 750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:43319 every 8 connection(s)
I20260811 01:45:37.359637 677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:45:37.363795 751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:37.369930 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 677
I20260811 01:45:37.370142 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:45:37.371065 22361 external_mini_cluster.cc:1428] Setting key fcc0e8676347ff82449f721b14deb96d
I20260811 01:45:37.376343 751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e: Bootstrap starting.
I20260811 01:45:37.380328 751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:37.381852 751 log.cc:824] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e: Log is configured to *not* fsync() on all Append() calls
I20260811 01:45:37.385432 751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e: No bootstrap required, opened a new log
I20260811 01:45:37.386802 754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:37.373608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45422 (local address 127.21.214.126:43319)
0811 01:45:37.374765 (+ 1157us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:37.374777 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:37.374824 (+ 47us) server_negotiation.cc:407] Connection header received
0811 01:45:37.376638 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:37.376673 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:37.377235 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:37.377650 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:37.378435 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:37.379853 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:37.381904 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:37.382847 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:37.383079 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:37.384348 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:37.384356 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:37.384371 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:45:37.384945 (+ 574us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:37.385036 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:45:37.385059 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:37.385216 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:37.385467 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 01:45:37.385801 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":705,"thread_start_us":456,"threads_started":1}
I20260811 01:45:37.393733 751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118d948343064be4800d37d062b1e82e" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43319 } }
I20260811 01:45:37.394268 751 raft_consensus.cc:374] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:37.394364 751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 118d948343064be4800d37d062b1e82e, State: Initialized, Role: FOLLOWER
I20260811 01:45:37.394989 751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118d948343064be4800d37d062b1e82e" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43319 } }
I20260811 01:45:37.395247 751 raft_consensus.cc:388] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:45:37.395428 751 raft_consensus.cc:482] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:45:37.395632 751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:37.398396 751 raft_consensus.cc:504] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118d948343064be4800d37d062b1e82e" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43319 } }
I20260811 01:45:37.398909 751 leader_election.cc:302] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 118d948343064be4800d37d062b1e82e; no voters:
I20260811 01:45:37.399482 751 leader_election.cc:288] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:45:37.399665 756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:45:37.400447 756 raft_consensus.cc:686] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e [term 1 LEADER]: Becoming Leader. State: Replica: 118d948343064be4800d37d062b1e82e, State: Running, Role: LEADER
I20260811 01:45:37.401029 756 consensus_queue.cc:237] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118d948343064be4800d37d062b1e82e" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43319 } }
I20260811 01:45:37.401826 751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:45:37.404059 757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "118d948343064be4800d37d062b1e82e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118d948343064be4800d37d062b1e82e" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43319 } } }
I20260811 01:45:37.404546 757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e [sys.catalog]: This master's current role is: LEADER
I20260811 01:45:37.404150 758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 118d948343064be4800d37d062b1e82e. Latest consensus state: current_term: 1 leader_uuid: "118d948343064be4800d37d062b1e82e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118d948343064be4800d37d062b1e82e" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43319 } } }
I20260811 01:45:37.405435 758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e [sys.catalog]: This master's current role is: LEADER
I20260811 01:45:37.407032 764 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:45:37.412803 764 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:45:37.423170 764 catalog_manager.cc:1347] Generated new cluster ID: ed1819be48534ca7ab7ac555bb048615
I20260811 01:45:37.423321 764 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:45:37.444280 764 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:45:37.445792 764 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:45:37.456584 764 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 118d948343064be4800d37d062b1e82e: Generated new TSK 0
I20260811 01:45:37.457597 764 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:45:37.467533 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:43319
--builtin_ntp_servers=127.21.214.84:36487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 01:45:37.829682 778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:37.830006 778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:37.830161 778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:37.838078 778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:37.838204 778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:37.838374 778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:45:37.847996 778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:36487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:43319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:37.849548 778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:37.851555 778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:37.865460 783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:37.865374 786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:37.865447 788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:37.866250 778 server_base.cc:1034] running on GCE node
I20260811 01:45:37.866984 778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:37.868022 778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:37.869253 778 hybrid_clock.cc:648] HybridClock initialized: now 1786412737869176 us; error 76 us; skew 500 ppm
I20260811 01:45:37.869596 778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:45:37.871950 778 webserver.cc:460] Webserver started at http://127.21.214.65:33353/ using document root <none> and password file <none>
I20260811 01:45:37.872721 778 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:37.872845 778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:37.873281 778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:37.876333 778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "3e5f68b4c9e64a9191918ac6d3709cd9"
format_stamp: "Formatted at 2026-08-11 01:45:37 on dist-test-slave-9gcw"
server_key: "639f7684ce1f015cc12eec3d30d9e06a"
server_key_iv: "62d08398484ec9b1e9e86ce061480fd9"
server_key_version: "encryptionkey@0"
I20260811 01:45:37.877251 778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "3e5f68b4c9e64a9191918ac6d3709cd9"
format_stamp: "Formatted at 2026-08-11 01:45:37 on dist-test-slave-9gcw"
server_key: "639f7684ce1f015cc12eec3d30d9e06a"
server_key_iv: "62d08398484ec9b1e9e86ce061480fd9"
server_key_version: "encryptionkey@0"
I20260811 01:45:37.882932 778 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 01:45:37.886827 795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:37.888260 778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 01:45:37.888477 778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "3e5f68b4c9e64a9191918ac6d3709cd9"
format_stamp: "Formatted at 2026-08-11 01:45:37 on dist-test-slave-9gcw"
server_key: "639f7684ce1f015cc12eec3d30d9e06a"
server_key_iv: "62d08398484ec9b1e9e86ce061480fd9"
server_key_version: "encryptionkey@0"
I20260811 01:45:37.888727 778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:37.901832 778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:37.902747 778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:37.903095 778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:37.904129 778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:45:37.905769 778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:37.905897 778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:37.906028 778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:37.906453 778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:37.940979 778 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:43533
I20260811 01:45:37.941037 924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:43533 every 8 connection(s)
I20260811 01:45:37.942425 778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:45:37.950151 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 778
I20260811 01:45:37.950376 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:45:37.951045 22361 external_mini_cluster.cc:1428] Setting key 49b55caee4352b76eb04c6171af3ca40
I20260811 01:45:37.953868 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:43319
--builtin_ntp_servers=127.21.214.84:36487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 01:45:37.956568 927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:37.945225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:48919 (local address 127.21.214.126:43319)
0811 01:45:37.945727 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:37.945732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:37.947112 (+ 1380us) server_negotiation.cc:407] Connection header received
0811 01:45:37.949003 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:37.949008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:37.949111 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:37.949267 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:37.950465 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:37.951293 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:37.953226 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:37.953648 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:37.953790 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:37.955476 (+ 1686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:37.955481 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:37.955484 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:45:37.955502 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:37.955545 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:45:37.955557 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:37.955786 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:37.956062 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 01:45:37.956202 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":241,"threads_started":1}
I20260811 01:45:37.957864 928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:37.945873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:43319 (local address 127.21.214.65:48919)
0811 01:45:37.946917 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:37.946970 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:37.948699 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:37.949519 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:37.949527 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:37.949636 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:37.950255 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:37.950272 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:37.951446 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:37.951452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:37.953026 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:37.953043 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:37.953889 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:37.953898 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:37.954237 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:37.955019 (+ 782us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:45:37.955022 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:37.955068 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:45:37.955169 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:37.955784 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:37.955798 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:37.955809 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 01:45:37.956043 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 01:45:37.956530 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":312,"threads_started":1}
I20260811 01:45:37.959669 925 heartbeater.cc:347] Connected to a master server at 127.21.214.126:43319
I20260811 01:45:37.960014 925 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:37.960861 925 heartbeater.cc:511] Master 127.21.214.126:43319 requested a full tablet report, sending...
I20260811 01:45:37.963562 711 ts_manager.cc:194] Registered new tserver with Master: 3e5f68b4c9e64a9191918ac6d3709cd9 (127.21.214.65:43533)
I20260811 01:45:37.965700 711 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.214.65:48919
W20260811 01:45:38.314339 930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:38.314654 930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:38.314807 930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:38.322301 930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:38.322417 930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:38.322590 930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:45:38.332051 930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:36487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:43319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:38.333575 930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:38.335505 930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:38.350358 938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:38.350319 935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:38.350414 936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:38.351280 930 server_base.cc:1034] running on GCE node
I20260811 01:45:38.351967 930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:38.352993 930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:38.354211 930 hybrid_clock.cc:648] HybridClock initialized: now 1786412738354165 us; error 59 us; skew 500 ppm
I20260811 01:45:38.354573 930 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:45:38.356982 930 webserver.cc:460] Webserver started at http://127.21.214.66:44279/ using document root <none> and password file <none>
I20260811 01:45:38.357919 930 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:38.358057 930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:38.358458 930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:38.362452 930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "c1ecd90d614641268eb1b6e49a9ae42c"
format_stamp: "Formatted at 2026-08-11 01:45:38 on dist-test-slave-9gcw"
server_key: "925cba844cda592c4e9b47ee04e1b552"
server_key_iv: "d2d9d43967b3cd4e877755cb8cae2ec2"
server_key_version: "encryptionkey@0"
I20260811 01:45:38.363452 930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "c1ecd90d614641268eb1b6e49a9ae42c"
format_stamp: "Formatted at 2026-08-11 01:45:38 on dist-test-slave-9gcw"
server_key: "925cba844cda592c4e9b47ee04e1b552"
server_key_iv: "d2d9d43967b3cd4e877755cb8cae2ec2"
server_key_version: "encryptionkey@0"
I20260811 01:45:38.370369 930 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 01:45:38.374923 946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:38.376614 930 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 01:45:38.376806 930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "c1ecd90d614641268eb1b6e49a9ae42c"
format_stamp: "Formatted at 2026-08-11 01:45:38 on dist-test-slave-9gcw"
server_key: "925cba844cda592c4e9b47ee04e1b552"
server_key_iv: "d2d9d43967b3cd4e877755cb8cae2ec2"
server_key_version: "encryptionkey@0"
I20260811 01:45:38.377036 930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:38.398955 930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:38.399823 930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:38.400094 930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:38.401088 930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:45:38.402766 930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:38.402858 930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:38.402953 930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:38.403009 930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:38.439855 930 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:33811
I20260811 01:45:38.439909 1072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:33811 every 8 connection(s)
I20260811 01:45:38.441344 930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:45:38.446816 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 930
I20260811 01:45:38.447005 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:45:38.447471 22361 external_mini_cluster.cc:1428] Setting key b87690ae66f0730664b16dc42ecb9f78
I20260811 01:45:38.450546 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:43319
--builtin_ntp_servers=127.21.214.84:36487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 01:45:38.455001 927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:38.444025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:40305 (local address 127.21.214.126:43319)
0811 01:45:38.444261 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:38.444270 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:38.445809 (+ 1539us) server_negotiation.cc:407] Connection header received
0811 01:45:38.447724 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:38.447732 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:38.447830 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:38.447956 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:38.449333 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:38.450274 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:38.452021 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:38.452422 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:38.452581 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:38.454119 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:38.454124 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:38.454127 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:45:38.454147 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:38.454182 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:45:38.454194 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:38.454302 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:38.454550 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 01:45:38.454658 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 01:45:38.456256 1075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:38.444644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:43319 (local address 127.21.214.66:40305)
0811 01:45:38.445630 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:38.445669 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:38.447406 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:38.448307 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:38.448318 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:38.448455 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:38.449062 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:38.449080 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:38.450433 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:38.450438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:38.451864 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:38.451877 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:38.452675 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:38.452682 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:38.452935 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:38.453763 (+ 828us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:45:38.453766 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:38.453811 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:45:38.453902 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:38.454300 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:38.454315 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:38.454326 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 01:45:38.454522 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 01:45:38.454973 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":273,"threads_started":1}
I20260811 01:45:38.458024 1073 heartbeater.cc:347] Connected to a master server at 127.21.214.126:43319
I20260811 01:45:38.458397 1073 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:38.459264 1073 heartbeater.cc:511] Master 127.21.214.126:43319 requested a full tablet report, sending...
I20260811 01:45:38.460844 711 ts_manager.cc:194] Registered new tserver with Master: c1ecd90d614641268eb1b6e49a9ae42c (127.21.214.66:33811)
I20260811 01:45:38.461725 711 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.214.66:40305
W20260811 01:45:38.811566 1076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:38.811879 1076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:38.812028 1076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:38.819581 1076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:38.819710 1076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:38.819895 1076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:45:38.829558 1076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:36487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:43319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:38.831027 1076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:38.832943 1076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:38.847055 1084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:38.847091 1082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:38.847162 1086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:38.848080 1076 server_base.cc:1034] running on GCE node
I20260811 01:45:38.848755 1076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:38.849898 1076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:38.851166 1076 hybrid_clock.cc:648] HybridClock initialized: now 1786412738851108 us; error 44 us; skew 500 ppm
I20260811 01:45:38.851548 1076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:45:38.854091 1076 webserver.cc:460] Webserver started at http://127.21.214.67:34963/ using document root <none> and password file <none>
I20260811 01:45:38.854863 1076 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:38.855008 1076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:38.855387 1076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:38.858646 1076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "b42d67d1e90c4b4b851bc159e1d791f8"
format_stamp: "Formatted at 2026-08-11 01:45:38 on dist-test-slave-9gcw"
server_key: "b21a369379ca6984bf040e77b4cc3329"
server_key_iv: "299c17520eb00d4b8aa176bc56646d97"
server_key_version: "encryptionkey@0"
I20260811 01:45:38.859498 1076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "b42d67d1e90c4b4b851bc159e1d791f8"
format_stamp: "Formatted at 2026-08-11 01:45:38 on dist-test-slave-9gcw"
server_key: "b21a369379ca6984bf040e77b4cc3329"
server_key_iv: "299c17520eb00d4b8aa176bc56646d97"
server_key_version: "encryptionkey@0"
I20260811 01:45:38.865792 1076 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 01:45:38.869820 1092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:38.871389 1076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 01:45:38.871659 1076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "b42d67d1e90c4b4b851bc159e1d791f8"
format_stamp: "Formatted at 2026-08-11 01:45:38 on dist-test-slave-9gcw"
server_key: "b21a369379ca6984bf040e77b4cc3329"
server_key_iv: "299c17520eb00d4b8aa176bc56646d97"
server_key_version: "encryptionkey@0"
I20260811 01:45:38.871928 1076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:38.906015 1076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:38.907105 1076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:38.907452 1076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:38.908594 1076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:45:38.910490 1076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:38.910620 1076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:38.910776 1076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:38.910868 1076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:38.950906 1076 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:40889
I20260811 01:45:38.951017 1221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:40889 every 8 connection(s)
I20260811 01:45:38.952212 1076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:45:38.952842 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 1076
I20260811 01:45:38.953019 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:45:38.953534 22361 external_mini_cluster.cc:1428] Setting key 98301cb953e043ae952e245d9ee61903
I20260811 01:45:38.965739 927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:38.955255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:38685 (local address 127.21.214.126:43319)
0811 01:45:38.955490 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:38.955495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:38.957175 (+ 1680us) server_negotiation.cc:407] Connection header received
0811 01:45:38.959018 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:38.959023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:38.959111 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:38.959229 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:38.960324 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:38.961089 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:38.962759 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:38.963210 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:38.963361 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:38.964817 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:38.964821 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:38.964825 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:45:38.964844 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:38.964875 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:45:38.964888 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:38.964985 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:38.965263 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 01:45:38.965370 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 01:45:38.966779 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:38.955925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:43319 (local address 127.21.214.67:38685)
0811 01:45:38.957001 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:38.957051 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:38.958752 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:38.959473 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:38.959482 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:38.959572 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:38.960146 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:38.960174 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:38.961313 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:38.961319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:38.962610 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:38.962623 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:38.963457 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:38.963464 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:38.963720 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:38.964468 (+ 748us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:45:38.964471 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:38.964510 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:45:38.964589 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:38.964986 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:38.964999 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:38.965009 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 01:45:38.965253 (+ 244us) client_negotiation.cc:240] Negotiation successful
0811 01:45:38.965670 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":303,"threads_started":1}
I20260811 01:45:38.968628 1222 heartbeater.cc:347] Connected to a master server at 127.21.214.126:43319
I20260811 01:45:38.968991 1222 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:38.969950 925 heartbeater.cc:503] Master 127.21.214.126:43319 was elected leader, sending a full tablet report...
I20260811 01:45:38.970101 1222 heartbeater.cc:511] Master 127.21.214.126:43319 requested a full tablet report, sending...
I20260811 01:45:38.971729 709 ts_manager.cc:194] Registered new tserver with Master: b42d67d1e90c4b4b851bc159e1d791f8 (127.21.214.67:40889)
I20260811 01:45:38.972553 709 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.214.67:38685
I20260811 01:45:38.982774 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 01:45:38.989384 927 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:38.987480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45432 (local address 127.21.214.126:43319)
0811 01:45:38.987826 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:38.987831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:38.988342 (+ 511us) server_negotiation.cc:407] Connection header received
0811 01:45:38.988416 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:38.988420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:38.988493 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:38.988594 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:38.988992 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45432: BlockingRecv error: recv got EOF from 127.0.0.1:45432 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 01:45:38.989776 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 778
I20260811 01:45:38.998939 1232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:38.990948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40016 (local address 127.21.214.65:43533)
0811 01:45:38.991582 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:38.991589 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:38.991617 (+ 28us) server_negotiation.cc:407] Connection header received
0811 01:45:38.991686 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:38.991692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:38.991918 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:38.992047 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:38.992627 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:38.993955 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:38.995391 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:38.996002 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:38.996107 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:38.997477 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:38.997485 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:38.997501 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:45:38.998107 (+ 606us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:38.998157 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:45:38.998178 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:38.998271 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:38.998444 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 01:45:38.998563 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":311,"threads_started":1}
I20260811 01:45:39.000296 919 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:39.087014 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 778
I20260811 01:45:39.109516 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 930
I20260811 01:45:39.117951 1234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:39.110418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52612 (local address 127.21.214.66:33811)
0811 01:45:39.111033 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:39.111041 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:39.111070 (+ 29us) server_negotiation.cc:407] Connection header received
0811 01:45:39.111131 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:39.111138 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:39.111380 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:39.111558 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:39.111997 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:39.113053 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:39.114376 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:39.114973 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:39.115114 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:39.116325 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:39.116334 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:39.116350 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:45:39.116985 (+ 635us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:39.117035 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:45:39.117056 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:39.117213 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:39.117414 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 01:45:39.117581 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":340,"threads_started":1}
I20260811 01:45:39.119360 1068 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:39.209218 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 930
I20260811 01:45:39.230576 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 1076
I20260811 01:45:39.239908 1236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:39.231654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44314 (local address 127.21.214.67:40889)
0811 01:45:39.232330 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:39.232338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:39.232371 (+ 33us) server_negotiation.cc:407] Connection header received
0811 01:45:39.232438 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:39.232444 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:39.232664 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:39.232818 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:39.233297 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:39.234411 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:39.236308 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:39.236948 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:39.237052 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:39.238316 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:39.238325 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:39.238340 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:45:39.238967 (+ 627us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:39.239028 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:45:39.239047 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:39.239139 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:39.239278 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 01:45:39.239434 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":363,"threads_started":1}
I20260811 01:45:39.241425 1216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:39.326830 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 1076
I20260811 01:45:39.349087 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 677
I20260811 01:45:39.350530 745 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:39.456128 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 677
2026-08-11T01:45:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2723 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T01:45:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:45:39Z Disabled control of system clock
I20260811 01:45:39.549855 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:43791
--webserver_interface=127.21.214.126
--webserver_port=0
--builtin_ntp_servers=127.21.214.84:46681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:43791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 01:45:39.908982 1242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:39.909428 1242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:39.909616 1242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:39.917114 1242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:45:39.917327 1242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:39.917425 1242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:39.917511 1242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:45:39.917599 1242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:45:39.927521 1242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:46681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:43791
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:43791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:39.929062 1242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:39.930986 1242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:39.944217 1248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:39.944121 1249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:39.944170 1251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:39.945187 1242 server_base.cc:1034] running on GCE node
I20260811 01:45:39.946025 1242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:39.947535 1242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:39.948789 1242 hybrid_clock.cc:648] HybridClock initialized: now 1786412739948705 us; error 56 us; skew 500 ppm
I20260811 01:45:39.949307 1242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:45:39.951905 1242 webserver.cc:460] Webserver started at http://127.21.214.126:41565/ using document root <none> and password file <none>
I20260811 01:45:39.952725 1242 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:39.952831 1242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:39.953306 1242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:39.956516 1242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "60450bcfa0b74b98a0a3729c275b0b03"
format_stamp: "Formatted at 2026-08-11 01:45:39 on dist-test-slave-9gcw"
server_key: "b0aa4527722de7cfdb9c16d622fd216c"
server_key_iv: "ed20a96f7e903bf2298e0f35edf09369"
server_key_version: "encryptionkey@0"
I20260811 01:45:39.957499 1242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "60450bcfa0b74b98a0a3729c275b0b03"
format_stamp: "Formatted at 2026-08-11 01:45:39 on dist-test-slave-9gcw"
server_key: "b0aa4527722de7cfdb9c16d622fd216c"
server_key_iv: "ed20a96f7e903bf2298e0f35edf09369"
server_key_version: "encryptionkey@0"
I20260811 01:45:39.963513 1242 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260811 01:45:39.967550 1258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:39.969375 1242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:45:39.969594 1242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "60450bcfa0b74b98a0a3729c275b0b03"
format_stamp: "Formatted at 2026-08-11 01:45:39 on dist-test-slave-9gcw"
server_key: "b0aa4527722de7cfdb9c16d622fd216c"
server_key_iv: "ed20a96f7e903bf2298e0f35edf09369"
server_key_version: "encryptionkey@0"
I20260811 01:45:39.969866 1242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:39.976028 1242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:39.977212 1242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:39.977583 1242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:39.998219 1242 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:43791
I20260811 01:45:39.998317 1315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:43791 every 8 connection(s)
I20260811 01:45:39.999847 1242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:45:40.000595 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 1242
I20260811 01:45:40.000880 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:45:40.001822 22361 external_mini_cluster.cc:1428] Setting key 9a806f0d5807cde5f1b63cfc08d70b46
I20260811 01:45:40.004653 1316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:40.011116 1319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:40.004153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54660 (local address 127.21.214.126:43791)
0811 01:45:40.005388 (+ 1235us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:40.005400 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:40.005448 (+ 48us) server_negotiation.cc:407] Connection header received
0811 01:45:40.007123 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:40.007149 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:40.007644 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:40.008004 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:40.009351 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:40.009358 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:40.009374 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:45:40.009389 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:40.009472 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:45:40.009497 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:40.009684 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:40.009813 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 01:45:40.010141 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":406,"threads_started":1}
I20260811 01:45:40.017890 1316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03: Bootstrap starting.
I20260811 01:45:40.021082 1316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:40.022528 1316 log.cc:824] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03: Log is configured to *not* fsync() on all Append() calls
I20260811 01:45:40.025432 1316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03: No bootstrap required, opened a new log
I20260811 01:45:40.033048 1316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60450bcfa0b74b98a0a3729c275b0b03" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43791 } }
I20260811 01:45:40.033550 1316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:40.033625 1316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60450bcfa0b74b98a0a3729c275b0b03, State: Initialized, Role: FOLLOWER
I20260811 01:45:40.034324 1316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60450bcfa0b74b98a0a3729c275b0b03" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43791 } }
I20260811 01:45:40.034629 1316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:45:40.034775 1316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:45:40.035056 1316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:40.037439 1316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60450bcfa0b74b98a0a3729c275b0b03" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43791 } }
I20260811 01:45:40.037938 1316 leader_election.cc:302] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60450bcfa0b74b98a0a3729c275b0b03; no voters:
I20260811 01:45:40.038581 1316 leader_election.cc:288] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:45:40.038753 1322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:45:40.039894 1322 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03 [term 1 LEADER]: Becoming Leader. State: Replica: 60450bcfa0b74b98a0a3729c275b0b03, State: Running, Role: LEADER
I20260811 01:45:40.040515 1322 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60450bcfa0b74b98a0a3729c275b0b03" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43791 } }
I20260811 01:45:40.041116 1316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:45:40.045831 1323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60450bcfa0b74b98a0a3729c275b0b03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60450bcfa0b74b98a0a3729c275b0b03" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43791 } } }
I20260811 01:45:40.046002 1324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60450bcfa0b74b98a0a3729c275b0b03. Latest consensus state: current_term: 1 leader_uuid: "60450bcfa0b74b98a0a3729c275b0b03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60450bcfa0b74b98a0a3729c275b0b03" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43791 } } }
I20260811 01:45:40.046352 1324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03 [sys.catalog]: This master's current role is: LEADER
I20260811 01:45:40.046357 1323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03 [sys.catalog]: This master's current role is: LEADER
I20260811 01:45:40.049320 1332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:45:40.054141 1332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:45:40.063433 1332 catalog_manager.cc:1347] Generated new cluster ID: 64d9cc1227824482a56eab4bf1b4c8ab
I20260811 01:45:40.063588 1332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:45:40.087461 1332 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:45:40.087961 1332 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:45:40.095233 1332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60450bcfa0b74b98a0a3729c275b0b03: Generated new TSK 0
I20260811 01:45:40.096251 1332 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:45:40.102864 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:43791
--builtin_ntp_servers=127.21.214.84:46681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 01:45:40.470443 1344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:40.470884 1344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:40.471076 1344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:40.479118 1344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:40.479245 1344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:40.479453 1344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:45:40.489408 1344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:46681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:43791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:40.490845 1344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:40.492854 1344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:40.507392 1350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:40.507308 1351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:40.507364 1353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:40.508669 1344 server_base.cc:1034] running on GCE node
I20260811 01:45:40.509413 1344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:40.510589 1344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:40.511852 1344 hybrid_clock.cc:648] HybridClock initialized: now 1786412740511797 us; error 68 us; skew 500 ppm
I20260811 01:45:40.512210 1344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:45:40.514804 1344 webserver.cc:460] Webserver started at http://127.21.214.65:45263/ using document root <none> and password file <none>
I20260811 01:45:40.515622 1344 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:40.515780 1344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:40.516151 1344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:40.519508 1344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "d0d68463862142809226beb661138c69"
format_stamp: "Formatted at 2026-08-11 01:45:40 on dist-test-slave-9gcw"
server_key: "8e0c84615a38f215fa86fa61d2da7e9b"
server_key_iv: "88a921dcf8e08187d7ccaa9e1bc7deb9"
server_key_version: "encryptionkey@0"
I20260811 01:45:40.520459 1344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "d0d68463862142809226beb661138c69"
format_stamp: "Formatted at 2026-08-11 01:45:40 on dist-test-slave-9gcw"
server_key: "8e0c84615a38f215fa86fa61d2da7e9b"
server_key_iv: "88a921dcf8e08187d7ccaa9e1bc7deb9"
server_key_version: "encryptionkey@0"
I20260811 01:45:40.526319 1344 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 01:45:40.530258 1359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:40.531826 1344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 01:45:40.532021 1344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "d0d68463862142809226beb661138c69"
format_stamp: "Formatted at 2026-08-11 01:45:40 on dist-test-slave-9gcw"
server_key: "8e0c84615a38f215fa86fa61d2da7e9b"
server_key_iv: "88a921dcf8e08187d7ccaa9e1bc7deb9"
server_key_version: "encryptionkey@0"
I20260811 01:45:40.532270 1344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:40.538497 1344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:40.539399 1344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:40.539685 1344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:40.540694 1344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:45:40.542383 1344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:40.542515 1344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:40.542649 1344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:40.542716 1344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:40.578835 1344 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:42911
I20260811 01:45:40.578945 1484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:42911 every 8 connection(s)
I20260811 01:45:40.580309 1344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:45:40.584527 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 1344
I20260811 01:45:40.584779 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:45:40.585786 22361 external_mini_cluster.cc:1428] Setting key a426ae4b7012d83fd0acd04bf8f054b1
I20260811 01:45:40.589349 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:43791
--builtin_ntp_servers=127.21.214.84:46681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 01:45:40.590214 1488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:40.583147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:46697 (local address 127.21.214.126:43791)
0811 01:45:40.583638 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:40.583646 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:40.585257 (+ 1611us) server_negotiation.cc:407] Connection header received
0811 01:45:40.587552 (+ 2295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:40.587557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:40.587631 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:40.587772 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:40.589462 (+ 1690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:40.589467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:40.589470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:45:40.589506 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:40.589561 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:45:40.589580 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:40.589705 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:40.589940 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 01:45:40.589993 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":228,"threads_started":1}
I20260811 01:45:40.591706 1491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:40.583795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:43791 (local address 127.21.214.65:46697)
0811 01:45:40.585008 (+ 1213us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:40.585062 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:40.587254 (+ 2192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:40.588049 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:40.588061 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:40.588234 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:40.589013 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:45:40.589016 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:40.589101 (+ 85us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:45:40.589261 (+ 160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:40.589703 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:40.589717 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:40.589729 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 01:45:40.590024 (+ 295us) client_negotiation.cc:240] Negotiation successful
0811 01:45:40.590336 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":350,"threads_started":1}
I20260811 01:45:40.593465 1485 heartbeater.cc:347] Connected to a master server at 127.21.214.126:43791
I20260811 01:45:40.593822 1485 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:40.594635 1485 heartbeater.cc:511] Master 127.21.214.126:43791 requested a full tablet report, sending...
I20260811 01:45:40.597018 1277 ts_manager.cc:194] Registered new tserver with Master: d0d68463862142809226beb661138c69 (127.21.214.65:42911)
W20260811 01:45:40.953435 1492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:40.953763 1492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:40.953904 1492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:40.961740 1492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:40.961863 1492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:40.962180 1492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:45:40.972366 1492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:46681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:43791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:40.973980 1492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:40.976063 1492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:40.991125 1498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:40.991097 1499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:40.991156 1501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:40.992489 1492 server_base.cc:1034] running on GCE node
I20260811 01:45:40.993287 1492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:40.994401 1492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:40.995661 1492 hybrid_clock.cc:648] HybridClock initialized: now 1786412740995606 us; error 65 us; skew 500 ppm
I20260811 01:45:40.996047 1492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:45:40.999048 1492 webserver.cc:460] Webserver started at http://127.21.214.66:33631/ using document root <none> and password file <none>
I20260811 01:45:40.999993 1492 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:41.000160 1492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:41.000571 1492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:41.004792 1492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "f75bac253a144d67b3dcc42994f91acd"
format_stamp: "Formatted at 2026-08-11 01:45:40 on dist-test-slave-9gcw"
server_key: "1aff868a65aea696e725a67ae286ec5a"
server_key_iv: "529b9d3804f7a7cafd0c599feeb00dc6"
server_key_version: "encryptionkey@0"
I20260811 01:45:41.005954 1492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "f75bac253a144d67b3dcc42994f91acd"
format_stamp: "Formatted at 2026-08-11 01:45:40 on dist-test-slave-9gcw"
server_key: "1aff868a65aea696e725a67ae286ec5a"
server_key_iv: "529b9d3804f7a7cafd0c599feeb00dc6"
server_key_version: "encryptionkey@0"
I20260811 01:45:41.013398 1492 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260811 01:45:41.018436 1509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:41.020298 1492 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260811 01:45:41.020489 1492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "f75bac253a144d67b3dcc42994f91acd"
format_stamp: "Formatted at 2026-08-11 01:45:40 on dist-test-slave-9gcw"
server_key: "1aff868a65aea696e725a67ae286ec5a"
server_key_iv: "529b9d3804f7a7cafd0c599feeb00dc6"
server_key_version: "encryptionkey@0"
I20260811 01:45:41.020742 1492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:41.028316 1492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:41.029397 1492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:41.029711 1492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:41.030783 1492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:45:41.032394 1492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:41.032532 1492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:41.032678 1492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:41.032769 1492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:41.071198 1492 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:35887
I20260811 01:45:41.071309 1633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:35887 every 8 connection(s)
I20260811 01:45:41.072674 1492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:45:41.082301 1488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:41.075954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:47021 (local address 127.21.214.126:43791)
0811 01:45:41.076189 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:41.076196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:41.077867 (+ 1671us) server_negotiation.cc:407] Connection header received
0811 01:45:41.079746 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:41.079750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:41.079830 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:41.079925 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:41.081446 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:41.081450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:41.081454 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:45:41.081471 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:41.081504 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:45:41.081517 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:41.081628 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:41.081843 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 01:45:41.081943 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 01:45:41.082432 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 1492
I20260811 01:45:41.082657 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:45:41.083207 22361 external_mini_cluster.cc:1428] Setting key 30d5aca04f848cbccd0f8c50c8acc670
I20260811 01:45:41.083511 1636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:41.076636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:43791 (local address 127.21.214.66:47021)
0811 01:45:41.077700 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:41.077739 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:41.079453 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:41.080156 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:41.080164 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:41.080264 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:41.081061 (+ 797us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:45:41.081063 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:41.081104 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:45:41.081263 (+ 159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:41.081611 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:41.081619 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:41.081625 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 01:45:41.081866 (+ 241us) client_negotiation.cc:240] Negotiation successful
0811 01:45:41.082171 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":289,"threads_started":1}
I20260811 01:45:41.085422 1634 heartbeater.cc:347] Connected to a master server at 127.21.214.126:43791
I20260811 01:45:41.085775 1634 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:41.086337 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:43791
--builtin_ntp_servers=127.21.214.84:46681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 01:45:41.086445 1634 heartbeater.cc:511] Master 127.21.214.126:43791 requested a full tablet report, sending...
I20260811 01:45:41.088008 1277 ts_manager.cc:194] Registered new tserver with Master: f75bac253a144d67b3dcc42994f91acd (127.21.214.66:35887)
W20260811 01:45:41.465945 1638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:41.466329 1638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:41.466511 1638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:41.474103 1638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:41.474290 1638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:41.474475 1638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:45:41.484553 1638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:46681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:43791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:41.486035 1638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:41.487943 1638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:41.501838 1644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:41.502141 1645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:41.501917 1647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:41.506904 1638 server_base.cc:1034] running on GCE node
I20260811 01:45:41.507620 1638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:41.508632 1638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:41.509895 1638 hybrid_clock.cc:648] HybridClock initialized: now 1786412741509844 us; error 48 us; skew 500 ppm
I20260811 01:45:41.510257 1638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:45:41.512924 1638 webserver.cc:460] Webserver started at http://127.21.214.67:34853/ using document root <none> and password file <none>
I20260811 01:45:41.513871 1638 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:41.514012 1638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:41.514362 1638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:41.517614 1638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "af0fd0f2cd3a4d3ca9b1012143578a64"
format_stamp: "Formatted at 2026-08-11 01:45:41 on dist-test-slave-9gcw"
server_key: "1b434f4c78ef2d952c45bde4495ef11f"
server_key_iv: "36fac09613930d1ce0778a5e8fad4084"
server_key_version: "encryptionkey@0"
I20260811 01:45:41.518478 1638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "af0fd0f2cd3a4d3ca9b1012143578a64"
format_stamp: "Formatted at 2026-08-11 01:45:41 on dist-test-slave-9gcw"
server_key: "1b434f4c78ef2d952c45bde4495ef11f"
server_key_iv: "36fac09613930d1ce0778a5e8fad4084"
server_key_version: "encryptionkey@0"
I20260811 01:45:41.524257 1638 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 01:45:41.528189 1653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:41.529809 1638 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:45:41.530007 1638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "af0fd0f2cd3a4d3ca9b1012143578a64"
format_stamp: "Formatted at 2026-08-11 01:45:41 on dist-test-slave-9gcw"
server_key: "1b434f4c78ef2d952c45bde4495ef11f"
server_key_iv: "36fac09613930d1ce0778a5e8fad4084"
server_key_version: "encryptionkey@0"
I20260811 01:45:41.530251 1638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:41.536325 1638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:41.537348 1638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:41.537633 1638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:41.538610 1638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:45:41.540114 1638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:41.540241 1638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:41.540380 1638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:41.540470 1638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:41.580768 1638 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:43167
I20260811 01:45:41.580825 1777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:43167 every 8 connection(s)
I20260811 01:45:41.582424 1638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:45:41.588371 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 1638
I20260811 01:45:41.588595 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:45:41.589048 22361 external_mini_cluster.cc:1428] Setting key 3169656652c507bf066f97ce6374db35
I20260811 01:45:41.592867 1780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:41.585867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:56271 (local address 127.21.214.126:43791)
0811 01:45:41.586421 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:41.586426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:41.587679 (+ 1253us) server_negotiation.cc:407] Connection header received
0811 01:45:41.590226 (+ 2547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:41.590232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:41.590325 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:41.590456 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:41.592068 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:41.592074 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:41.592078 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:45:41.592098 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:41.592152 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:45:41.592169 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:41.592299 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:41.592535 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 01:45:41.592608 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":282,"threads_started":1}
I20260811 01:45:41.593999 1781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:41.586459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:43791 (local address 127.21.214.67:56271)
0811 01:45:41.587482 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:41.587523 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:41.589814 (+ 2291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:41.590796 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:41.590805 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:41.590908 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:41.591714 (+ 806us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:45:41.591717 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:41.591762 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:45:41.591861 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:41.592289 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:41.592302 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:41.592314 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 01:45:41.592601 (+ 287us) client_negotiation.cc:240] Negotiation successful
0811 01:45:41.592956 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":300,"threads_started":1}
I20260811 01:45:41.595638 1778 heartbeater.cc:347] Connected to a master server at 127.21.214.126:43791
I20260811 01:45:41.595959 1778 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:41.596637 1778 heartbeater.cc:511] Master 127.21.214.126:43791 requested a full tablet report, sending...
I20260811 01:45:41.598203 1277 ts_manager.cc:194] Registered new tserver with Master: af0fd0f2cd3a4d3ca9b1012143578a64 (127.21.214.67:43167)
I20260811 01:45:41.600548 1485 heartbeater.cc:503] Master 127.21.214.126:43791 was elected leader, sending a full tablet report...
I20260811 01:45:41.605316 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 01:45:41.611768 1780 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:45:41.610078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54676 (local address 127.21.214.126:43791)
0811 01:45:41.610286 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:41.610291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:41.610857 (+ 566us) server_negotiation.cc:407] Connection header received
0811 01:45:41.610998 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:41.611002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:41.611053 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:41.611142 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:41.611474 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54676: BlockingRecv error: recv got EOF from 127.0.0.1:54676 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 01:45:41.612596 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 1344
I20260811 01:45:41.616799 1790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:41.613702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51318 (local address 127.21.214.65:42911)
0811 01:45:41.614323 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:41.614339 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:41.614365 (+ 26us) server_negotiation.cc:407] Connection header received
0811 01:45:41.614428 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:41.614434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:41.614650 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:41.614838 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:41.616011 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:41.616018 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:41.616030 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:45:41.616042 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:41.616084 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:45:41.616097 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:41.616310 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:41.616385 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 01:45:41.616492 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":285,"threads_started":1}
I20260811 01:45:41.618212 1479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:41.720821 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 1344
I20260811 01:45:41.748272 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 1492
I20260811 01:45:41.752660 1792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:41.749442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53576 (local address 127.21.214.66:35887)
0811 01:45:41.750192 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:41.750199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:41.750231 (+ 32us) server_negotiation.cc:407] Connection header received
0811 01:45:41.750305 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:41.750311 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:41.750546 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:41.750685 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:41.751814 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:41.751821 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:41.751834 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:45:41.751855 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:41.751902 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:45:41.751919 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:41.752050 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:41.752169 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 01:45:41.752314 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":425,"threads_started":1}
I20260811 01:45:41.754189 1629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:41.837632 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 1492
I20260811 01:45:41.859074 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 1638
I20260811 01:45:41.863716 1795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:41.860304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57886 (local address 127.21.214.67:43167)
0811 01:45:41.861311 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:41.861326 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:41.861358 (+ 32us) server_negotiation.cc:407] Connection header received
0811 01:45:41.861427 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:41.861434 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:41.861666 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:41.861812 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:41.862935 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:41.862942 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:41.862953 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:45:41.862969 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:41.863007 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:45:41.863021 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:41.863166 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:41.863225 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 01:45:41.863307 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":760,"thread_start_us":610,"threads_started":1}
I20260811 01:45:41.864976 1773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:41.950922 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 1638
I20260811 01:45:41.973719 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 1242
I20260811 01:45:41.975144 1311 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:42.083261 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 1242
2026-08-11T01:45:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2626 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:45:42 dist-test-slave-9gcw krb5kdc[1800](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:45:42 dist-test-slave-9gcw krb5kdc[1800](info): set up 2 sockets
Aug 11 01:45:42 dist-test-slave-9gcw krb5kdc[1800](info): commencing operation
krb5kdc: starting...
W20260811 01:45:43.367476 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:45:43 dist-test-slave-9gcw krb5kdc[1800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:45:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:45:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:45:43.687296 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:40965
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:40965
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:45:44.054057 1815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:44.054520 1815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:44.054733 1815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:44.064921 1815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:45:44.065124 1815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:44.065279 1815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:44.065413 1815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:45:44.065531 1815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:45:44.077406 1815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:40965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:40965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:44.079034 1815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:44.080535 1815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:44.093815 1823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:44.093922 1824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:44.093945 1827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:44.095458 1815 server_base.cc:1034] running on GCE node
I20260811 01:45:44.096441 1815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:44.098157 1815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:44.099436 1815 hybrid_clock.cc:648] HybridClock initialized: now 1786412744099380 us; error 46 us; skew 500 ppm
I20260811 01:45:44.099846 1815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:45:44 dist-test-slave-9gcw krb5kdc[1800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412744, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:45:44.110966 1815 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:45:44.112828 1815 webserver.cc:460] Webserver started at http://127.21.214.126:45147/ using document root <none> and password file <none>
I20260811 01:45:44.113729 1815 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:44.113852 1815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:44.114156 1815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:44.117756 1815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "4df0e06a1fa24a27a0c26866c5e4aa0c"
format_stamp: "Formatted at 2026-08-11 01:45:44 on dist-test-slave-9gcw"
server_key: "0c5a4c38145f7b03cb7c999760899844"
server_key_iv: "f6c47f7556918052014ac0bcc5e7f1c4"
server_key_version: "encryptionkey@0"
I20260811 01:45:44.118678 1815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "4df0e06a1fa24a27a0c26866c5e4aa0c"
format_stamp: "Formatted at 2026-08-11 01:45:44 on dist-test-slave-9gcw"
server_key: "0c5a4c38145f7b03cb7c999760899844"
server_key_iv: "f6c47f7556918052014ac0bcc5e7f1c4"
server_key_version: "encryptionkey@0"
I20260811 01:45:44.125418 1815 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 01:45:44.130015 1835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:44.131894 1815 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 01:45:44.132144 1815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "4df0e06a1fa24a27a0c26866c5e4aa0c"
format_stamp: "Formatted at 2026-08-11 01:45:44 on dist-test-slave-9gcw"
server_key: "0c5a4c38145f7b03cb7c999760899844"
server_key_iv: "f6c47f7556918052014ac0bcc5e7f1c4"
server_key_version: "encryptionkey@0"
I20260811 01:45:44.132368 1815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:44.149463 1815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:44.154253 1815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:44.154590 1815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:44.175129 1815 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:40965
I20260811 01:45:44.175164 1892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:40965 every 8 connection(s)
I20260811 01:45:44.176656 1815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:45:44.178898 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 1815
I20260811 01:45:44.179157 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:45:44.179845 22361 external_mini_cluster.cc:1428] Setting key 267066123e755129e156b3bd4aa3b26e
I20260811 01:45:44.180860 1894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 01:45:44 dist-test-slave-9gcw krb5kdc[1800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:45:44.196301 1894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c: Bootstrap starting.
I20260811 01:45:44.199810 1894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:44.200939 1894 log.cc:824] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c: Log is configured to *not* fsync() on all Append() calls
I20260811 01:45:44.204308 1894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c: No bootstrap required, opened a new log
I20260811 01:45:44.204414 1897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:44.181920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57262 (local address 127.21.214.126:40965)
0811 01:45:44.182701 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:44.182713 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:44.182756 (+ 43us) server_negotiation.cc:407] Connection header received
0811 01:45:44.184349 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:44.184372 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:44.184781 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:44.185209 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:44.187178 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:44.188573 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:44.190456 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:44.191154 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:44.191301 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:44.196544 (+ 5243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:44.196554 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:44.196571 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:44.196614 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:44.199791 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:44.200571 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:44.200578 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:44.200585 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:44.200757 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:44.201244 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:44.201249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:44.201252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:44.202040 (+ 788us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:44.202419 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:44.202882 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 01:45:44.203363 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":347,"threads_started":1}
I20260811 01:45:44.212981 1894 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4df0e06a1fa24a27a0c26866c5e4aa0c" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40965 } }
I20260811 01:45:44.213483 1894 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:44.213598 1894 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df0e06a1fa24a27a0c26866c5e4aa0c, State: Initialized, Role: FOLLOWER
I20260811 01:45:44.214179 1894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4df0e06a1fa24a27a0c26866c5e4aa0c" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40965 } }
I20260811 01:45:44.214438 1894 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:45:44.214581 1894 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:45:44.214782 1894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:44.217439 1894 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4df0e06a1fa24a27a0c26866c5e4aa0c" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40965 } }
I20260811 01:45:44.217970 1894 leader_election.cc:302] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4df0e06a1fa24a27a0c26866c5e4aa0c; no voters:
I20260811 01:45:44.218633 1894 leader_election.cc:288] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:45:44.218786 1899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:45:44.219666 1899 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c [term 1 LEADER]: Becoming Leader. State: Replica: 4df0e06a1fa24a27a0c26866c5e4aa0c, State: Running, Role: LEADER
I20260811 01:45:44.220336 1899 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4df0e06a1fa24a27a0c26866c5e4aa0c" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40965 } }
I20260811 01:45:44.221113 1894 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:45:44.223481 1900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4df0e06a1fa24a27a0c26866c5e4aa0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4df0e06a1fa24a27a0c26866c5e4aa0c" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40965 } } }
I20260811 01:45:44.224213 1900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c [sys.catalog]: This master's current role is: LEADER
I20260811 01:45:44.223799 1901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4df0e06a1fa24a27a0c26866c5e4aa0c. Latest consensus state: current_term: 1 leader_uuid: "4df0e06a1fa24a27a0c26866c5e4aa0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4df0e06a1fa24a27a0c26866c5e4aa0c" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40965 } } }
I20260811 01:45:44.224869 1901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c [sys.catalog]: This master's current role is: LEADER
I20260811 01:45:44.225828 1912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:45:44.231134 1912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:45:44.241605 1912 catalog_manager.cc:1347] Generated new cluster ID: 79ca7689329c47baaea14f6218ae3e3b
I20260811 01:45:44.241745 1912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:45:44.259229 1912 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:45:44.261274 1912 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:45:44.275460 1912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c: Generated new TSK 0
I20260811 01:45:44.276753 1912 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
I20260811 01:45:44.357277 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40965
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:45:44.717782 1924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:44.718140 1924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:44.718317 1924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:44.726629 1924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:44.726799 1924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:44.726997 1924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:45:44.738389 1924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:44.740058 1924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:44.741868 1924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:44.757728 1929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:44.757766 1932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:44.757699 1930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:44.758783 1924 server_base.cc:1034] running on GCE node
I20260811 01:45:44.759459 1924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:44.760509 1924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:44.761816 1924 hybrid_clock.cc:648] HybridClock initialized: now 1786412744761766 us; error 61 us; skew 500 ppm
I20260811 01:45:44.762228 1924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:45:44 dist-test-slave-9gcw krb5kdc[1800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412744, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:45:44.766237 1924 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:45:44.768018 1924 webserver.cc:460] Webserver started at http://127.21.214.65:40727/ using document root <none> and password file <none>
I20260811 01:45:44.768909 1924 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:44.769068 1924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:44.769438 1924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:44.772642 1924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "eb33e685abd040ddafaa74944d34d9d7"
format_stamp: "Formatted at 2026-08-11 01:45:44 on dist-test-slave-9gcw"
server_key: "8626580bcc842aebc1f3386371454a20"
server_key_iv: "ed747e94eaeecbc873fd0d0ea4c51b71"
server_key_version: "encryptionkey@0"
I20260811 01:45:44.773556 1924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "eb33e685abd040ddafaa74944d34d9d7"
format_stamp: "Formatted at 2026-08-11 01:45:44 on dist-test-slave-9gcw"
server_key: "8626580bcc842aebc1f3386371454a20"
server_key_iv: "ed747e94eaeecbc873fd0d0ea4c51b71"
server_key_version: "encryptionkey@0"
I20260811 01:45:44.779642 1924 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 01:45:44.783743 1939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:44.786031 1924 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 01:45:44.786216 1924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "eb33e685abd040ddafaa74944d34d9d7"
format_stamp: "Formatted at 2026-08-11 01:45:44 on dist-test-slave-9gcw"
server_key: "8626580bcc842aebc1f3386371454a20"
server_key_iv: "ed747e94eaeecbc873fd0d0ea4c51b71"
server_key_version: "encryptionkey@0"
I20260811 01:45:44.786412 1924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:44.809433 1924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:44.814731 1924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:44.815029 1924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:44.816211 1924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:45:44.817775 1924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:44.817858 1924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:44.817970 1924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:44.818019 1924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:44.855827 1924 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:38951
I20260811 01:45:44.855855 2067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:38951 every 8 connection(s)
I20260811 01:45:44.857482 1924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:45:44.860792 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 1924
I20260811 01:45:44.861043 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:45:44.862004 22361 external_mini_cluster.cc:1428] Setting key ac0c7221e6ae00c1ebd912495b6f600a
Aug 11 01:45:44 dist-test-slave-9gcw krb5kdc[1800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412744, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:45:44.881371 2070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:44.860339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:52495 (local address 127.21.214.126:40965)
0811 01:45:44.860979 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:44.860987 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:44.862385 (+ 1398us) server_negotiation.cc:407] Connection header received
0811 01:45:44.864828 (+ 2443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:44.864834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:44.864938 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:44.865068 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:44.866872 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:44.867696 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:44.869628 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:44.870094 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:44.870261 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:44.874446 (+ 4185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:44.874452 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:44.874455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:44.874494 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:44.877626 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:44.878552 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:44.878558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:44.878561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:44.878701 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:44.879252 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:44.879258 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:44.879260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:44.879567 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:44.879774 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:44.880621 (+ 847us) server_negotiation.cc:299] Negotiation successful
0811 01:45:44.880895 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":344,"threads_started":1}
I20260811 01:45:44.882493 2071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:44.861101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40965 (local address 127.21.214.65:52495)
0811 01:45:44.862158 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:44.862231 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:44.864509 (+ 2278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:44.865391 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:44.865400 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:44.866001 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:44.866667 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:44.866688 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:44.867885 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:44.867899 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:44.869447 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:44.869462 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:44.870389 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:44.870398 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:44.870669 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:44.871498 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:44.871529 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:44.874145 (+ 2616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:44.877810 (+ 3665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:44.877821 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:44.877843 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:44.878384 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:44.878853 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:44.878859 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:44.878864 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:44.879100 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:44.879727 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:44.879737 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:44.880169 (+ 432us) client_negotiation.cc:769] Sending connection context
0811 01:45:44.880480 (+ 311us) client_negotiation.cc:240] Negotiation successful
0811 01:45:44.881168 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"thread_start_us":414,"threads_started":1}
I20260811 01:45:44.884406 2068 heartbeater.cc:347] Connected to a master server at 127.21.214.126:40965
I20260811 01:45:44.884781 2068 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:44.885895 2068 heartbeater.cc:511] Master 127.21.214.126:40965 requested a full tablet report, sending...
I20260811 01:45:44.888814 1852 ts_manager.cc:194] Registered new tserver with Master: eb33e685abd040ddafaa74944d34d9d7 (127.21.214.65:38951)
I20260811 01:45:44.891266 1852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:52495
WARNING: no policy specified for kudu/127.21.214.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
I20260811 01:45:44.932952 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40965
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:45:45.295013 2074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:45.295387 2074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:45.295559 2074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:45.303086 2074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:45.303261 2074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:45.303452 2074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:45:45.314069 2074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:45.315549 2074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:45.316983 2074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:45.331890 2080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:45.331964 2081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:45.332448 2083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:45.338121 2074 server_base.cc:1034] running on GCE node
I20260811 01:45:45.338841 2074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:45.340022 2074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:45.341368 2074 hybrid_clock.cc:648] HybridClock initialized: now 1786412745341324 us; error 50 us; skew 500 ppm
I20260811 01:45:45.341735 2074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:45:45 dist-test-slave-9gcw krb5kdc[1800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412745, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:45:45.345530 2074 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:45:45.347236 2074 webserver.cc:460] Webserver started at http://127.21.214.66:44843/ using document root <none> and password file <none>
I20260811 01:45:45.348006 2074 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:45.348160 2074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:45.348495 2074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:45.351686 2074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "0218545be2c14428a2a362512be824ab"
format_stamp: "Formatted at 2026-08-11 01:45:45 on dist-test-slave-9gcw"
server_key: "14602bda81f8c4c906609ca1cac58241"
server_key_iv: "282915d94d4095fc92486e06d3ea8407"
server_key_version: "encryptionkey@0"
I20260811 01:45:45.352507 2074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "0218545be2c14428a2a362512be824ab"
format_stamp: "Formatted at 2026-08-11 01:45:45 on dist-test-slave-9gcw"
server_key: "14602bda81f8c4c906609ca1cac58241"
server_key_iv: "282915d94d4095fc92486e06d3ea8407"
server_key_version: "encryptionkey@0"
I20260811 01:45:45.358135 2074 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 01:45:45.361984 2090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:45.363484 2074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:45:45.363689 2074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "0218545be2c14428a2a362512be824ab"
format_stamp: "Formatted at 2026-08-11 01:45:45 on dist-test-slave-9gcw"
server_key: "14602bda81f8c4c906609ca1cac58241"
server_key_iv: "282915d94d4095fc92486e06d3ea8407"
server_key_version: "encryptionkey@0"
I20260811 01:45:45.363925 2074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:45.387696 2074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:45.392328 2074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:45.392663 2074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:45.393779 2074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:45:45.395289 2074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:45.395426 2074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:45.395586 2074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:45.395663 2074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:45.436803 2074 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:45477
I20260811 01:45:45.436908 2219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:45477 every 8 connection(s)
I20260811 01:45:45.438288 2074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:45:45.445623 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 2074
I20260811 01:45:45.445851 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:45:45.446419 22361 external_mini_cluster.cc:1428] Setting key 3e4a01f0abd2eee32c4ab68be0efa86b
Aug 11 01:45:45 dist-test-slave-9gcw krb5kdc[1800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412745, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:45:45.461752 2223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:45.441063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:44607 (local address 127.21.214.126:40965)
0811 01:45:45.441576 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:45.441583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:45.442605 (+ 1022us) server_negotiation.cc:407] Connection header received
0811 01:45:45.444562 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:45.444568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:45.444670 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:45.444793 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:45.447013 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:45.447902 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:45.450313 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:45.450722 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:45.450831 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:45.455300 (+ 4469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:45.455305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:45.455309 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:45.455356 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:45.458239 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:45.459104 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:45.459109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:45.459112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:45.459249 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:45.459753 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:45.459757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:45.459759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:45.460065 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:45.460235 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:45.461063 (+ 828us) server_negotiation.cc:299] Negotiation successful
0811 01:45:45.461352 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":300,"threads_started":1}
I20260811 01:45:45.462903 2224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:45.441684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40965 (local address 127.21.214.66:44607)
0811 01:45:45.442421 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:45.442483 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:45.444272 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:45.445045 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:45.445054 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:45.445928 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:45.446803 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:45.446838 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:45.448069 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:45.448073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:45.450146 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:45.450165 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:45.450968 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:45.450980 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:45.451338 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:45.452150 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:45.452186 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:45.455012 (+ 2826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:45.458417 (+ 3405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:45.458429 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:45.458450 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:45.458947 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:45.459379 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:45.459384 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:45.459388 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:45.459611 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:45.460211 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:45.460220 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:45.460647 (+ 427us) client_negotiation.cc:769] Sending connection context
0811 01:45:45.460951 (+ 304us) client_negotiation.cc:240] Negotiation successful
0811 01:45:45.461628 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":274,"threads_started":1}
I20260811 01:45:45.464586 2220 heartbeater.cc:347] Connected to a master server at 127.21.214.126:40965
I20260811 01:45:45.464962 2220 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:45.466012 2220 heartbeater.cc:511] Master 127.21.214.126:40965 requested a full tablet report, sending...
I20260811 01:45:45.468016 1852 ts_manager.cc:194] Registered new tserver with Master: 0218545be2c14428a2a362512be824ab (127.21.214.66:45477)
I20260811 01:45:45.469403 1852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:44607
WARNING: no policy specified for kudu/127.21.214.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
I20260811 01:45:45.516750 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40965
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:45:45.878610 2227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:45:45.878916 2227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:45:45.879014 2227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:45:45.886475 2227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:45:45.886598 2227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:45:45.886785 2227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:45:45.895201 2068 heartbeater.cc:503] Master 127.21.214.126:40965 was elected leader, sending a full tablet report...
I20260811 01:45:45.897825 2227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:45:45.899252 2227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:45:45.900650 2227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:45:45.915050 2232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:45.915117 2233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:45:45.915359 2235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:45:45.916177 2227 server_base.cc:1034] running on GCE node
I20260811 01:45:45.916863 2227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:45:45.917989 2227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:45:45.919234 2227 hybrid_clock.cc:648] HybridClock initialized: now 1786412745919170 us; error 60 us; skew 500 ppm
I20260811 01:45:45.919581 2227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:45:45 dist-test-slave-9gcw krb5kdc[1800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412745, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:45:45.923314 2227 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:45:45.925336 2227 webserver.cc:460] Webserver started at http://127.21.214.67:41653/ using document root <none> and password file <none>
I20260811 01:45:45.926267 2227 fs_manager.cc:359] Metadata directory not provided
I20260811 01:45:45.926430 2227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:45:45.926865 2227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:45:45.930423 2227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "cd269bc740074ec991cfffb738e6b929"
format_stamp: "Formatted at 2026-08-11 01:45:45 on dist-test-slave-9gcw"
server_key: "7af2b276f1e8cf9788adeb70982791df"
server_key_iv: "afd0987f0f48d2ea73de1a13ebe59003"
server_key_version: "encryptionkey@0"
I20260811 01:45:45.931325 2227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "cd269bc740074ec991cfffb738e6b929"
format_stamp: "Formatted at 2026-08-11 01:45:45 on dist-test-slave-9gcw"
server_key: "7af2b276f1e8cf9788adeb70982791df"
server_key_iv: "afd0987f0f48d2ea73de1a13ebe59003"
server_key_version: "encryptionkey@0"
I20260811 01:45:45.937556 2227 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 01:45:45.941473 2243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:45.942947 2227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:45:45.943146 2227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "cd269bc740074ec991cfffb738e6b929"
format_stamp: "Formatted at 2026-08-11 01:45:45 on dist-test-slave-9gcw"
server_key: "7af2b276f1e8cf9788adeb70982791df"
server_key_iv: "afd0987f0f48d2ea73de1a13ebe59003"
server_key_version: "encryptionkey@0"
I20260811 01:45:45.943379 2227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:45:45.969796 2227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:45:45.974517 2227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:45:45.974864 2227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:45:45.975921 2227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:45:45.977425 2227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:45:45.977540 2227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:45.977690 2227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:45:45.977763 2227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:45:46.013688 2227 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:45269
I20260811 01:45:46.013793 2368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:45269 every 8 connection(s)
I20260811 01:45:46.015111 2227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:45:46.019924 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 2227
I20260811 01:45:46.020226 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:45:46.020848 22361 external_mini_cluster.cc:1428] Setting key 50d8985cdbc2e5bda287c15ab20dbbf5
Aug 11 01:45:46 dist-test-slave-9gcw krb5kdc[1800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412745, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:45:46.039098 2371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:46.018012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:57171 (local address 127.21.214.126:40965)
0811 01:45:46.018501 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:46.018505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:46.019486 (+ 981us) server_negotiation.cc:407] Connection header received
0811 01:45:46.022667 (+ 3181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:46.022675 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:46.022770 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:46.022904 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:46.024941 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.026012 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.027775 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.028243 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.028394 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:46.032525 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:46.032531 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:46.032534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:46.032570 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:46.035435 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:46.036257 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:46.036264 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:46.036267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:46.036421 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:46.037051 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:46.037058 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:46.037061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:46.037451 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:46.037632 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:46.038436 (+ 804us) server_negotiation.cc:299] Negotiation successful
0811 01:45:46.038678 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":285,"threads_started":1}
I20260811 01:45:46.040236 2372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:46.018496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40965 (local address 127.21.214.67:57171)
0811 01:45:46.019232 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:46.019273 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:46.022268 (+ 2995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:46.023230 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:46.023244 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:46.023935 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:45:46.024756 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:46.024777 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.026194 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.026202 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:46.027588 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:46.027614 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.028514 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.028522 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:46.028793 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:46.029627 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:45:46.029654 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:45:46.032259 (+ 2605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:45:46.035604 (+ 3345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:46.035614 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:46.035633 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:46.036114 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:46.036599 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:45:46.036604 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:45:46.036607 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:45:46.036922 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:45:46.037615 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:45:46.037625 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:45:46.038034 (+ 409us) client_negotiation.cc:769] Sending connection context
0811 01:45:46.038344 (+ 310us) client_negotiation.cc:240] Negotiation successful
0811 01:45:46.038966 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":283,"threads_started":1}
I20260811 01:45:46.041949 2369 heartbeater.cc:347] Connected to a master server at 127.21.214.126:40965
I20260811 01:45:46.042246 2369 heartbeater.cc:464] Registering TS with master...
I20260811 01:45:46.043005 2369 heartbeater.cc:511] Master 127.21.214.126:40965 requested a full tablet report, sending...
I20260811 01:45:46.044582 1852 ts_manager.cc:194] Registered new tserver with Master: cd269bc740074ec991cfffb738e6b929 (127.21.214.67:45269)
I20260811 01:45:46.045462 1852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:57171
I20260811 01:45:46.049965 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:45:46.069564 2371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:46.054718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57276 (local address 127.21.214.126:40965)
0811 01:45:46.054925 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:46.054930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:46.055484 (+ 554us) server_negotiation.cc:407] Connection header received
0811 01:45:46.055631 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:46.055635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:46.055715 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:46.055810 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:46.057418 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.058151 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.059442 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.059833 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.059933 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:46.062526 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:46.062532 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:46.062536 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:46.062574 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:46.065353 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:46.066163 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:46.066168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:46.066171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:46.066378 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:46.066829 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:46.066834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:46.066838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:46.067469 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:46.067780 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:46.068986 (+ 1206us) server_negotiation.cc:299] Negotiation successful
0811 01:45:46.069242 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 01:45:46.073972 1852 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:45:46.081111 1852 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:45:46.120409 2384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:46.094960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50604 (local address 127.21.214.66:45477)
0811 01:45:46.106751 (+ 11791us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:46.106762 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:46.107390 (+ 628us) server_negotiation.cc:407] Connection header received
0811 01:45:46.107575 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:46.107584 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:46.107867 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:46.108084 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:46.109402 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.111640 (+ 2238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.115972 (+ 4332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.118203 (+ 2231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.118366 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:46.118671 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:46.119787 (+ 1116us) server_negotiation.cc:299] Negotiation successful
0811 01:45:46.120010 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11547,"thread_start_us":11420,"threads_started":1}
I20260811 01:45:46.121534 2386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:46.095696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50874 (local address 127.21.214.65:38951)
0811 01:45:46.107916 (+ 12220us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:46.107927 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:46.107964 (+ 37us) server_negotiation.cc:407] Connection header received
0811 01:45:46.108095 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:46.108102 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:46.108377 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:46.108606 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:46.109694 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.111652 (+ 1958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.115983 (+ 4331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.118228 (+ 2245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.118416 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:46.118731 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:46.120874 (+ 2143us) server_negotiation.cc:299] Negotiation successful
0811 01:45:46.121071 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12022,"thread_start_us":12006,"threads_started":1}
I20260811 01:45:46.122126 2385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:46.094961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:38951 (local address 127.0.0.1:50874)
0811 01:45:46.106852 (+ 11891us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:46.106936 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:46.107230 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:46.108900 (+ 1670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:46.108905 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:46.108932 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:46.109434 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:46.109452 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.111890 (+ 2438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.111900 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:46.115703 (+ 3803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:46.115729 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.118968 (+ 3239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.118978 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:46.120015 (+ 1037us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:46.120058 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 01:45:46.120168 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 01:45:46.120400 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11567,"mutex_wait_us":725,"thread_start_us":11983,"threads_started":1}
I20260811 01:45:46.122303 2383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:46.095084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:45477 (local address 127.0.0.1:50604)
0811 01:45:46.107230 (+ 12146us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:46.107250 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:46.107388 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:46.108439 (+ 1051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:46.108447 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:46.108479 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:46.109089 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:46.109114 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.111892 (+ 2778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.111900 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:46.115707 (+ 3807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:46.115731 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.119019 (+ 3288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.119031 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:46.119398 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:46.119483 (+ 85us) client_negotiation.cc:769] Sending connection context
0811 01:45:46.120833 (+ 1350us) client_negotiation.cc:240] Negotiation successful
0811 01:45:46.120971 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12016,"mutex_wait_us":257,"thread_start_us":12418,"threads_started":1}
I20260811 01:45:46.126358 2145 tablet_service.cc:1467] Processing CreateTablet for tablet 04d3e5122c3142168f87e2c6b828cef5 (DEFAULT_TABLE table=test-table [id=3862d4adfba342a1a411627554efb9cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:46.126448 1996 tablet_service.cc:1467] Processing CreateTablet for tablet 04d3e5122c3142168f87e2c6b828cef5 (DEFAULT_TABLE table=test-table [id=3862d4adfba342a1a411627554efb9cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:46.128073 2145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04d3e5122c3142168f87e2c6b828cef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:46.128578 1996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04d3e5122c3142168f87e2c6b828cef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:46.129357 2387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:46.108615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:45269 (local address 127.0.0.1:55398)
0811 01:45:46.120233 (+ 11618us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:46.120252 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:46.120382 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:46.121359 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:46.121362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:46.121385 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:46.121804 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:46.121819 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.123921 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.123926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:46.126208 (+ 2282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:46.126225 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.128463 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.128470 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:46.128664 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:46.128701 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 01:45:46.128802 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 01:45:46.128956 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11472,"thread_start_us":11261,"threads_started":1}
I20260811 01:45:46.129837 2388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:46.109593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55398 (local address 127.21.214.67:45269)
0811 01:45:46.119872 (+ 10279us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:46.119885 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:46.120518 (+ 633us) server_negotiation.cc:407] Connection header received
0811 01:45:46.120780 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:46.120794 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:46.121112 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:46.121375 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:46.122010 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.123740 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.126430 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.127900 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.128038 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:46.128316 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:46.129292 (+ 976us) server_negotiation.cc:299] Negotiation successful
0811 01:45:46.129481 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10060,"thread_start_us":9849,"threads_started":1}
I20260811 01:45:46.133929 2296 tablet_service.cc:1467] Processing CreateTablet for tablet 04d3e5122c3142168f87e2c6b828cef5 (DEFAULT_TABLE table=test-table [id=3862d4adfba342a1a411627554efb9cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:45:46.135780 2296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04d3e5122c3142168f87e2c6b828cef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:45:46.142402 2389 tablet_bootstrap.cc:492] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7: Bootstrap starting.
I20260811 01:45:46.146023 2390 tablet_bootstrap.cc:492] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab: Bootstrap starting.
I20260811 01:45:46.147500 2389 tablet_bootstrap.cc:654] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:46.148751 2389 log.cc:824] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7: Log is configured to *not* fsync() on all Append() calls
I20260811 01:45:46.150810 2390 tablet_bootstrap.cc:654] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:46.151819 2389 tablet_bootstrap.cc:492] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7: No bootstrap required, opened a new log
I20260811 01:45:46.152209 2390 log.cc:824] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab: Log is configured to *not* fsync() on all Append() calls
I20260811 01:45:46.152235 2389 ts_tablet_manager.cc:1397] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 01:45:46.152509 2392 tablet_bootstrap.cc:492] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929: Bootstrap starting.
I20260811 01:45:46.156455 2392 tablet_bootstrap.cc:654] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929: Neither blocks nor log segments found. Creating new log.
I20260811 01:45:46.156700 2390 tablet_bootstrap.cc:492] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab: No bootstrap required, opened a new log
I20260811 01:45:46.157023 2390 ts_tablet_manager.cc:1397] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 01:45:46.157790 2392 log.cc:824] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929: Log is configured to *not* fsync() on all Append() calls
I20260811 01:45:46.161042 2392 tablet_bootstrap.cc:492] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929: No bootstrap required, opened a new log
I20260811 01:45:46.161407 2392 ts_tablet_manager.cc:1397] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260811 01:45:46.161908 2389 raft_consensus.cc:348] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0218545be2c14428a2a362512be824ab" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 45477 } } peers { permanent_uuid: "eb33e685abd040ddafaa74944d34d9d7" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38951 } } peers { permanent_uuid: "cd269bc740074ec991cfffb738e6b929" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45269 } }
I20260811 01:45:46.162345 2389 raft_consensus.cc:374] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:46.162443 2389 raft_consensus.cc:729] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb33e685abd040ddafaa74944d34d9d7, State: Initialized, Role: FOLLOWER
I20260811 01:45:46.163215 2389 consensus_queue.cc:260] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0218545be2c14428a2a362512be824ab" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 45477 } } peers { permanent_uuid: "eb33e685abd040ddafaa74944d34d9d7" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38951 } } peers { permanent_uuid: "cd269bc740074ec991cfffb738e6b929" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45269 } }
I20260811 01:45:46.165520 2389 ts_tablet_manager.cc:1428] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260811 01:45:46.167402 2390 raft_consensus.cc:348] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0218545be2c14428a2a362512be824ab" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 45477 } } peers { permanent_uuid: "eb33e685abd040ddafaa74944d34d9d7" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38951 } } peers { permanent_uuid: "cd269bc740074ec991cfffb738e6b929" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45269 } }
I20260811 01:45:46.167819 2390 raft_consensus.cc:374] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:46.167934 2390 raft_consensus.cc:729] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0218545be2c14428a2a362512be824ab, State: Initialized, Role: FOLLOWER
I20260811 01:45:46.168578 2390 consensus_queue.cc:260] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0218545be2c14428a2a362512be824ab" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 45477 } } peers { permanent_uuid: "eb33e685abd040ddafaa74944d34d9d7" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38951 } } peers { permanent_uuid: "cd269bc740074ec991cfffb738e6b929" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45269 } }
I20260811 01:45:46.170362 2220 heartbeater.cc:503] Master 127.21.214.126:40965 was elected leader, sending a full tablet report...
I20260811 01:45:46.170533 2390 ts_tablet_manager.cc:1428] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab: Time spent starting tablet: real 0.013s user 0.007s sys 0.007s
I20260811 01:45:46.172938 2392 raft_consensus.cc:348] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0218545be2c14428a2a362512be824ab" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 45477 } } peers { permanent_uuid: "eb33e685abd040ddafaa74944d34d9d7" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38951 } } peers { permanent_uuid: "cd269bc740074ec991cfffb738e6b929" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45269 } }
I20260811 01:45:46.173378 2392 raft_consensus.cc:374] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:45:46.173492 2392 raft_consensus.cc:729] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd269bc740074ec991cfffb738e6b929, State: Initialized, Role: FOLLOWER
I20260811 01:45:46.174257 2392 consensus_queue.cc:260] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0218545be2c14428a2a362512be824ab" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 45477 } } peers { permanent_uuid: "eb33e685abd040ddafaa74944d34d9d7" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38951 } } peers { permanent_uuid: "cd269bc740074ec991cfffb738e6b929" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45269 } }
I20260811 01:45:46.175751 2369 heartbeater.cc:503] Master 127.21.214.126:40965 was elected leader, sending a full tablet report...
I20260811 01:45:46.176378 2392 ts_tablet_manager.cc:1428] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929: Time spent starting tablet: real 0.015s user 0.012s sys 0.001s
W20260811 01:45:46.190377 2221 tablet.cc:2369] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:45:46.267030 2370 tablet.cc:2369] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:45:46.360410 2069 tablet.cc:2369] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:45:46.475296 2398 raft_consensus.cc:482] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:45:46.475749 2398 raft_consensus.cc:504] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0218545be2c14428a2a362512be824ab" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 45477 } } peers { permanent_uuid: "eb33e685abd040ddafaa74944d34d9d7" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38951 } } peers { permanent_uuid: "cd269bc740074ec991cfffb738e6b929" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45269 } }
I20260811 01:45:46.477594 2398 leader_election.cc:288] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0218545be2c14428a2a362512be824ab (127.21.214.66:45477), eb33e685abd040ddafaa74944d34d9d7 (127.21.214.65:38951)
I20260811 01:45:46.485726 2384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:46.478246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:48009 (local address 127.21.214.66:45477)
0811 01:45:46.478472 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:46.478479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:46.478505 (+ 26us) server_negotiation.cc:407] Connection header received
0811 01:45:46.478738 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:46.478745 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:46.478845 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:46.478965 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:46.479568 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.480542 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.483375 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.484625 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.484808 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:46.484890 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:46.485303 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 01:45:46.485441 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 01:45:46.485780 2372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:46.477925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:45477 (local address 127.21.214.67:48009)
0811 01:45:46.478315 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:46.478340 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:46.478499 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:46.479022 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:46.479028 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:46.479079 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:46.479344 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:46.479354 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.480712 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.480718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:46.483197 (+ 2479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:46.483214 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.484815 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.484819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:46.485022 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:46.485073 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 01:45:46.485266 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 01:45:46.485419 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":84}
I20260811 01:45:46.485857 2399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:46.478055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:38951 (local address 127.21.214.67:35083)
0811 01:45:46.478950 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0811 01:45:46.478973 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:45:46.479114 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:45:46.479479 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:45:46.479487 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:45:46.479519 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:45:46.479873 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:46.479882 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.481126 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.481132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:46.483461 (+ 2329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:45:46.483473 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.485090 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.485095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:45:46.485310 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:46.485342 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 01:45:46.485424 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 01:45:46.485542 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"thread_start_us":643,"threads_started":1}
I20260811 01:45:46.485877 2386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:46.478281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:35083 (local address 127.21.214.65:38951)
0811 01:45:46.478567 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:46.478576 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:46.479082 (+ 506us) server_negotiation.cc:407] Connection header received
0811 01:45:46.479260 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:46.479264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:46.479343 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:46.479418 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:45:46.480038 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.480965 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.483613 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.484878 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.485025 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:46.485074 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:46.485471 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 01:45:46.485632 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 01:45:46.487075 2168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04d3e5122c3142168f87e2c6b828cef5" candidate_uuid: "cd269bc740074ec991cfffb738e6b929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0218545be2c14428a2a362512be824ab" is_pre_election: true
I20260811 01:45:46.487075 2017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04d3e5122c3142168f87e2c6b828cef5" candidate_uuid: "cd269bc740074ec991cfffb738e6b929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb33e685abd040ddafaa74944d34d9d7" is_pre_election: true
I20260811 01:45:46.487711 2017 raft_consensus.cc:2454] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd269bc740074ec991cfffb738e6b929 in term 0.
I20260811 01:45:46.487711 2168 raft_consensus.cc:2454] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd269bc740074ec991cfffb738e6b929 in term 0.
I20260811 01:45:46.488479 2245 leader_election.cc:302] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0218545be2c14428a2a362512be824ab, cd269bc740074ec991cfffb738e6b929; no voters:
I20260811 01:45:46.488971 2398 raft_consensus.cc:2781] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:45:46.489163 2398 raft_consensus.cc:482] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:45:46.489300 2398 raft_consensus.cc:3037] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:46.492110 2398 raft_consensus.cc:504] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0218545be2c14428a2a362512be824ab" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 45477 } } peers { permanent_uuid: "eb33e685abd040ddafaa74944d34d9d7" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38951 } } peers { permanent_uuid: "cd269bc740074ec991cfffb738e6b929" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45269 } }
I20260811 01:45:46.492833 2398 leader_election.cc:288] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [CANDIDATE]: Term 1 election: Requested vote from peers 0218545be2c14428a2a362512be824ab (127.21.214.66:45477), eb33e685abd040ddafaa74944d34d9d7 (127.21.214.65:38951)
I20260811 01:45:46.493207 2168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04d3e5122c3142168f87e2c6b828cef5" candidate_uuid: "cd269bc740074ec991cfffb738e6b929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0218545be2c14428a2a362512be824ab"
I20260811 01:45:46.493439 2168 raft_consensus.cc:3037] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:46.493613 2017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04d3e5122c3142168f87e2c6b828cef5" candidate_uuid: "cd269bc740074ec991cfffb738e6b929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb33e685abd040ddafaa74944d34d9d7"
I20260811 01:45:46.493878 2017 raft_consensus.cc:3037] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:45:46.496798 2168 raft_consensus.cc:2454] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd269bc740074ec991cfffb738e6b929 in term 1.
I20260811 01:45:46.497143 2017 raft_consensus.cc:2454] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd269bc740074ec991cfffb738e6b929 in term 1.
I20260811 01:45:46.497294 2245 leader_election.cc:302] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0218545be2c14428a2a362512be824ab, cd269bc740074ec991cfffb738e6b929; no voters:
I20260811 01:45:46.497650 2398 raft_consensus.cc:2781] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:45:46.498147 2398 raft_consensus.cc:686] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [term 1 LEADER]: Becoming Leader. State: Replica: cd269bc740074ec991cfffb738e6b929, State: Running, Role: LEADER
I20260811 01:45:46.498646 2398 consensus_queue.cc:237] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0218545be2c14428a2a362512be824ab" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 45477 } } peers { permanent_uuid: "eb33e685abd040ddafaa74944d34d9d7" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38951 } } peers { permanent_uuid: "cd269bc740074ec991cfffb738e6b929" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45269 } }
I20260811 01:45:46.503283 1851 catalog_manager.cc:5495] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd269bc740074ec991cfffb738e6b929 (127.21.214.67). New cstate: current_term: 1 leader_uuid: "cd269bc740074ec991cfffb738e6b929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0218545be2c14428a2a362512be824ab" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 45477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb33e685abd040ddafaa74944d34d9d7" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd269bc740074ec991cfffb738e6b929" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45269 } health_report { overall_health: HEALTHY } } }
I20260811 01:45:46.592794 2388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:46.586929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55414 (local address 127.21.214.67:45269)
0811 01:45:46.587125 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:46.587129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:46.587510 (+ 381us) server_negotiation.cc:407] Connection header received
0811 01:45:46.587739 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:46.587743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:46.587840 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:46.587977 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:45:46.588377 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.589181 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.590621 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.591042 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.591135 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:46.591625 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:45:46.592258 (+ 633us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:45:46.592370 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:46.592478 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 01:45:46.592548 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 01:45:46.601096 2017 raft_consensus.cc:1261] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7 [term 1 FOLLOWER]: Refusing update from remote peer cd269bc740074ec991cfffb738e6b929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:45:46.601100 2168 raft_consensus.cc:1261] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab [term 1 FOLLOWER]: Refusing update from remote peer cd269bc740074ec991cfffb738e6b929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:45:46.602213 2400 consensus_queue.cc:1035] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb33e685abd040ddafaa74944d34d9d7" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:45:46.602597 2398 consensus_queue.cc:1035] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0218545be2c14428a2a362512be824ab" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 45477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:45:46.616554 2407 mvcc.cc:205] Tried to move back new op lower bound from 7317146610068201472 to 7317146609662267392. Current Snapshot: MvccSnapshot[applied={T|T < 7317146610068201472}]
I20260811 01:45:46.634405 1852 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:45:46.634769 1852 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:45:46.638906 1852 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4df0e06a1fa24a27a0c26866c5e4aa0c: Sending DeleteTablet for 3 replicas of tablet 04d3e5122c3142168f87e2c6b828cef5
I20260811 01:45:46.640425 1996 tablet_service.cc:1514] Processing DeleteTablet for tablet 04d3e5122c3142168f87e2c6b828cef5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:46 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:50874
I20260811 01:45:46.640407 2145 tablet_service.cc:1514] Processing DeleteTablet for tablet 04d3e5122c3142168f87e2c6b828cef5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:46 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:50604
I20260811 01:45:46.640741 2296 tablet_service.cc:1514] Processing DeleteTablet for tablet 04d3e5122c3142168f87e2c6b828cef5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:45:46 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:55398
I20260811 01:45:46.641733 2417 tablet_replica.cc:330] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7: stopping tablet replica
I20260811 01:45:46.642490 2417 raft_consensus.cc:2229] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:45:46.643309 2417 raft_consensus.cc:2258] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:46.644446 2418 tablet_replica.cc:330] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab: stopping tablet replica
I20260811 01:45:46.644980 2418 raft_consensus.cc:2229] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:45:46.645825 2418 raft_consensus.cc:2258] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:46.645116 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 1924
I20260811 01:45:46.648581 2417 ts_tablet_manager.cc:1905] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:46.655887 2419 tablet_replica.cc:330] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929: stopping tablet replica
I20260811 01:45:46.656497 2419 raft_consensus.cc:2229] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:45:46.657039 2418 ts_tablet_manager.cc:1905] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:46.657408 2419 raft_consensus.cc:2258] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:45:46.659989 2419 ts_tablet_manager.cc:1905] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:45:46.666520 2418 ts_tablet_manager.cc:1918] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:45:46.666860 2418 log.cc:1196] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/04d3e5122c3142168f87e2c6b828cef5
I20260811 01:45:46.667716 2418 ts_tablet_manager.cc:1939] T 04d3e5122c3142168f87e2c6b828cef5 P 0218545be2c14428a2a362512be824ab: Deleting consensus metadata
I20260811 01:45:46.669585 2417 ts_tablet_manager.cc:1918] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:45:46.669862 2417 log.cc:1196] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/04d3e5122c3142168f87e2c6b828cef5
I20260811 01:45:46.670526 2417 ts_tablet_manager.cc:1939] T 04d3e5122c3142168f87e2c6b828cef5 P eb33e685abd040ddafaa74944d34d9d7: Deleting consensus metadata
I20260811 01:45:46.670469 1837 catalog_manager.cc:4841] TS 0218545be2c14428a2a362512be824ab (127.21.214.66:45477): tablet 04d3e5122c3142168f87e2c6b828cef5 (table test-table [id=3862d4adfba342a1a411627554efb9cd]) successfully deleted
I20260811 01:45:46.671494 2419 ts_tablet_manager.cc:1918] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:45:46.671787 2419 log.cc:1196] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/04d3e5122c3142168f87e2c6b828cef5
Aug 11 01:45:46 dist-test-slave-9gcw krb5kdc[1800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:45:46.672508 2419 ts_tablet_manager.cc:1939] T 04d3e5122c3142168f87e2c6b828cef5 P cd269bc740074ec991cfffb738e6b929: Deleting consensus metadata
I20260811 01:45:46.673166 1836 catalog_manager.cc:4841] TS eb33e685abd040ddafaa74944d34d9d7 (127.21.214.65:38951): tablet 04d3e5122c3142168f87e2c6b828cef5 (table test-table [id=3862d4adfba342a1a411627554efb9cd]) successfully deleted
I20260811 01:45:46.674582 1836 catalog_manager.cc:4841] TS cd269bc740074ec991cfffb738e6b929 (127.21.214.67:45269): tablet 04d3e5122c3142168f87e2c6b828cef5 (table test-table [id=3862d4adfba342a1a411627554efb9cd]) successfully deleted
I20260811 01:45:46.680493 2386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:46.649192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50876 (local address 127.21.214.65:38951)
0811 01:45:46.649405 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:46.649413 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:46.659562 (+ 10149us) server_negotiation.cc:407] Connection header received
0811 01:45:46.660108 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:46.660125 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:46.660236 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:46.660327 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:46.662783 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.664071 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.666185 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.666904 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.667022 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:46.673470 (+ 6448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:46.673486 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:46.673508 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:46.673554 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:46.677363 (+ 3809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:46.678327 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:46.678335 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:46.678337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:46.678486 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:46.678913 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:46.678916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:46.678918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:46.679314 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:46.679550 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:46.679937 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 01:45:46.680130 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 01:45:46.681371 2063 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:46.774437 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 1924
I20260811 01:45:46.804963 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 2074
Aug 11 01:45:46 dist-test-slave-9gcw krb5kdc[1800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.66@KRBTEST.COM
I20260811 01:45:46.823033 2384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:46.805880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50612 (local address 127.21.214.66:45477)
0811 01:45:46.806063 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:46.806068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:46.806171 (+ 103us) server_negotiation.cc:407] Connection header received
0811 01:45:46.806369 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:46.806373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:46.806449 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:46.806535 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:46.808246 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.808965 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.810973 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.811421 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.811528 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:46.816795 (+ 5267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:46.816813 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:46.816831 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:46.816870 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:46.819912 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:46.820703 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:46.820713 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:46.820716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:46.820858 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:46.821339 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:46.821345 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:46.821350 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:46.821764 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:46.822105 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:46.822423 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 01:45:46.822663 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 01:45:46.823832 2214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:46.918860 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 2074
I20260811 01:45:46.945541 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 2227
Aug 11 01:45:46 dist-test-slave-9gcw krb5kdc[1800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.67@KRBTEST.COM
I20260811 01:45:46.965672 2388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:45:46.946579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55424 (local address 127.21.214.67:45269)
0811 01:45:46.946809 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 01:45:46.946815 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:45:46.946840 (+ 25us) server_negotiation.cc:407] Connection header received
0811 01:45:46.946912 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:45:46.946918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:45:46.947040 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:45:46.947181 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:45:46.949077 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.950351 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.951675 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:45:46.952407 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:45:46.952579 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:45:46.957521 (+ 4942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:45:46.957540 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:45:46.957564 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:45:46.957621 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:45:46.962193 (+ 4572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:46.963035 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:46.963046 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:46.963049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:46.963277 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:45:46.963680 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:45:46.963685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:45:46.963689 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:45:46.964210 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:45:46.964591 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0811 01:45:46.964916 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 01:45:46.965227 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 01:45:46.966645 2364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:47.097781 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 2227
I20260811 01:45:47.123239 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 1815
I20260811 01:45:47.124835 1888 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:45:47.239665 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 1815
2026-08-11T01:45:47Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (5155 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:45:47 dist-test-slave-9gcw krb5kdc[2430](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:45:47 dist-test-slave-9gcw krb5kdc[2430](info): set up 2 sockets
Aug 11 01:45:47 dist-test-slave-9gcw krb5kdc[2430](info): commencing operation
krb5kdc: starting...
W20260811 01:45:48.473084 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.001s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:45:48 dist-test-slave-9gcw krb5kdc[2430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:45:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:45:48Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.214.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.214.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.214.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/rangeradmin_127.21.214.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.214.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/rangeradmin_127.21.214.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.214.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.214.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.214.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/rangerlookup_127.21.214.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.214.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/rangerlookup_127.21.214.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.214.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.214.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.214.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/HTTP_127.21.214.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.214.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/HTTP_127.21.214.84@KRBTEST.COM.keytab.
I20260811 01:45:48.916625 22361 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskw71eKA/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/postgres -l logfile start
2026-08-11 01:45:51.769 UTC [2463] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 01:45:51.769 UTC [2463] LOG: listening on IPv4 address "127.21.214.84", port 50903
2026-08-11 01:45:51.774 UTC [2463] LOG: listening on Unix socket "/tmp/.s.PGSQL.50903"
2026-08-11 01:45:51.800 UTC [2465] LOG: database system was shut down at 2026-08-11 01:45:50 UTC
2026-08-11 01:45:51.804 UTC [2463] LOG: database system is ready to accept connections
I20260811 01:45:52.915479 22361 mini_postgres.cc:96] Postgres bound to 50903
2026-08-11 01:45:52.945 UTC [2476] FATAL: database "slave" does not exist
127.21.214.84:50903 - accepting connections
I20260811 01:45:52.946024 22361 mini_ranger.cc:161] Starting Ranger...
I20260811 01:45:52.983726 22361 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 01:45:53.559059 22361 mini_ranger.cc:87] Created ranger Postgres database
I20260811 01:45:53.559216 22361 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ranger-admin
2026-08-11 01:45:53,910 [I] DB FLAVOR :POSTGRES
2026-08-11 01:45:53,911 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 01:45:53,911 [I] Checking connection..
2026-08-11 01:45:53,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 01:45:54,190 [I] Checking connection passed.
2026-08-11 01:45:54,190 [I] --------- Verifying version history table ---------
2026-08-11 01:45:54,190 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 01:45:54,466 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 01:45:54,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 01:45:54,726 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 01:45:54,726 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 01:45:54,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 01:45:54.991 UTC [2552] WARNING: there is no transaction in progress
2026-08-11 01:45:54,999 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 01:45:54,999 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 01:45:55,241 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 01:45:55,242 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 01:45:55,242 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 01:45:55,511 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 01:45:55,773 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9gcw.c.gcp-upstream.internal','N') ;"
2026-08-11 01:45:56,032 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 01:45:56,290 [I] Table x_portal_user does not exist in database ranger
2026-08-11 01:45:56,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 01:45:56,590 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 01:45:56,590 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 01:45:56,861 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 01:45:57,117 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 01:45:57,117 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 01:45:58.155 UTC [2672] WARNING: there is no transaction in progress
2026-08-11 01:45:58.172 UTC [2672] WARNING: there is no transaction in progress
2026-08-11 01:45:58.191 UTC [2672] WARNING: there is no transaction in progress
2026-08-11 01:45:58.207 UTC [2672] WARNING: there is no transaction in progress
2026-08-11 01:45:58.223 UTC [2672] WARNING: there is no transaction in progress
2026-08-11 01:45:58.241 UTC [2672] WARNING: there is no transaction in progress
2026-08-11 01:45:58.300 UTC [2672] WARNING: there is no transaction in progress
2026-08-11 01:45:58.307 UTC [2672] WARNING: there is no transaction in progress
2026-08-11 01:45:58.323 UTC [2672] WARNING: there is no transaction in progress
2026-08-11 01:45:58.335 UTC [2672] WARNING: there is no transaction in progress
2026-08-11 01:45:58.346 UTC [2672] WARNING: there is no transaction in progress
2026-08-11 01:45:58.362 UTC [2672] WARNING: there is no transaction in progress
2026-08-11 01:45:58.373 UTC [2672] WARNING: there is no transaction in progress
2026-08-11 01:45:58.379 UTC [2672] WARNING: there is no transaction in progress
2026-08-11 01:45:58.388 UTC [2672] WARNING: there is no transaction in progress
2026-08-11 01:45:58.395 UTC [2672] WARNING: there is no transaction in progress
2026-08-11 01:45:59,108 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 01:45:59,109 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 01:45:59,380 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 01:45:59,380 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 01:45:59,647 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 01:45:59,648 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 01:45:59,914 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 01:45:59,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 01:46:00,186 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 01:46:00,462 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9gcw.c.gcp-upstream.internal';"
2026-08-11 01:46:00,714 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 01:46:00,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 01:46:00,989 [I] DB_PATCHES have already been applied
I20260811 01:46:00.996310 22361 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ranger-admin:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskw71eKA/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 01:46:00.996417 22361 mini_ranger.cc:193] Using host: 127.21.214.84
I20260811 01:46:01.019438 22361 mini_ranger.cc:227] Ranger admin URL: http://127.21.214.84:46823
Aug 11, 2026 1:46:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 1:46:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 1:46:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 1:46:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 1:46:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 1:46:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 1:46:02 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 01:46:02.333855 22361 mini_ranger.cc:160] Time spent starting Ranger: real 9.388s user 0.000s sys 0.074s
Aug 11, 2026 1:46:23 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 1:46:23 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 1:46:23 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 1:46:23 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 1:46:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 1:46:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 1:46:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 1:46:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 1:46:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 1:46:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 1:46:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 1:46:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 1:46:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 1:46:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 1:46:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 1:46:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 1:46:23 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 1:46:24 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 01:46:25.232278 22361 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.214.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.214.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.214.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/rangerkms_127.21.214.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.214.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/rangerkms_127.21.214.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.214.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.214.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.214.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/HTTP_127.21.214.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.214.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/HTTP_127.21.214.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 01:46:25 dist-test-slave-9gcw krb5kdc[2430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412785, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 01:46:25.463977 22361 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 01:46:25.494064 22361 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 01:46:26.495812 22361 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 01:46:26.496017 22361 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ranger-kms
I20260811 01:46:26.496136 22361 mini_ranger_kms.cc:227] Using postgres at 127.21.214.84:50903
2026-08-11 01:46:27,267 [I] DB FLAVOR :POSTGRES
2026-08-11 01:46:27,267 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 01:46:27,267 [I] Checking connection
2026-08-11 01:46:27,267 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 01:46:27,530 [I] connection success
2026-08-11 01:46:27,530 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 01:46:27,531 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 01:46:27,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 01:46:27,808 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 01:46:27,809 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 01:46:27,809 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 01:46:27,809 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:50903/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 01:46:28,125 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 01:46:28.656664 22361 mini_ranger_kms.cc:310] Created kms service
I20260811 01:46:28.858436 22361 mini_ranger_kms.cc:326] Created kudu user
I20260811 01:46:28.908034 22361 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 01:46:28 dist-test-slave-9gcw krb5kdc[2430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412785, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
I20260811 01:46:29.337993 22361 mini_ranger_kms.cc:384] Added ranger policy
I20260811 01:46:29.339709 22361 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ranger-kms:/tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 01:46:29.339809 22361 mini_ranger_kms.cc:240] Using host: 127.21.214.84
I20260811 01:46:29.366585 22361 mini_ranger_kms.cc:276] Ranger KMS PID: 2874
I20260811 01:46:29.366853 22361 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.214.84:36281
Aug 11, 2026 1:46:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 1:46:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 1:46:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 1:46:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 1:46:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 1:46:30 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 01:46:31 dist-test-slave-9gcw krb5kdc[2430](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786412791, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.214.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:46:31.962774 22361 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.499s user 0.145s sys 0.466s
I20260811 01:46:31.963194 22361 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 01:46:31.963451 22361 mini_ranger_kms.cc:401] 127.21.214.84:36281/kms/v1/keys
Aug 11 01:46:32 dist-test-slave-9gcw krb5kdc[2430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412791, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.214.84@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
Aug 11 01:46:32 dist-test-slave-9gcw krb5kdc[2430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412791, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.214.84@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
Aug 11 01:46:35 dist-test-slave-9gcw krb5kdc[2430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:46:35.338337 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:42791
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:33885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:42791
--ranger_config_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.214.84:36281/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:46:35.911227 2929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:46:35.911661 2929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:46:35.911871 2929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:46:35.920926 2929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:46:35.921180 2929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:46:35.921300 2929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:46:35.921485 2929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:46:35.921638 2929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:46:35.935953 2929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:33885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.214.84:36281/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:42791
--ranger_config_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:42791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:46:35.937754 2929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:46:35.939409 2929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:46:35.956178 2934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:46:35.956929 2938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:46:35.957098 2935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:46:35.958724 2929 server_base.cc:1034] running on GCE node
I20260811 01:46:35.959789 2929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:46:35.961735 2929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:46:35.963129 2929 hybrid_clock.cc:648] HybridClock initialized: now 1786412795962985 us; error 58 us; skew 500 ppm
I20260811 01:46:35.963753 2929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:46:35 dist-test-slave-9gcw krb5kdc[2430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412795, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:46:35.968465 2929 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:46:35.971012 2929 webserver.cc:460] Webserver started at http://127.21.214.126:41005/ using document root <none> and password file <none>
I20260811 01:46:35.971909 2929 fs_manager.cc:359] Metadata directory not provided
I20260811 01:46:35.972118 2929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:46:35.972555 2929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 01:46:35 dist-test-slave-9gcw krb5kdc[2430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412795, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
I20260811 01:46:36.032480 2929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "b3969bd8cd164a9486d9e8121124c37d"
format_stamp: "Formatted at 2026-08-11 01:46:36 on dist-test-slave-9gcw"
server_key: "0beafba54a734034324d337ad4bc1969"
server_key_iv: "0fb20d6f1d00a9f9f022a16d2891eb06"
server_key_version: "kuduclusterkey@0"
I20260811 01:46:36.033681 2929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "b3969bd8cd164a9486d9e8121124c37d"
format_stamp: "Formatted at 2026-08-11 01:46:36 on dist-test-slave-9gcw"
server_key: "0beafba54a734034324d337ad4bc1969"
server_key_iv: "0fb20d6f1d00a9f9f022a16d2891eb06"
server_key_version: "kuduclusterkey@0"
I20260811 01:46:36.041661 2929 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 01:46:36.061986 2945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:46:36.063930 2929 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260811 01:46:36.064248 2929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "b3969bd8cd164a9486d9e8121124c37d"
format_stamp: "Formatted at 2026-08-11 01:46:36 on dist-test-slave-9gcw"
server_key: "0beafba54a734034324d337ad4bc1969"
server_key_iv: "0fb20d6f1d00a9f9f022a16d2891eb06"
server_key_version: "kuduclusterkey@0"
I20260811 01:46:36.064617 2929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:46:36.087862 2929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:46:36.093516 2929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:46:36.094015 2929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:46:36.133833 2929 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:42791
I20260811 01:46:36.133952 2998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:42791 every 8 connection(s)
I20260811 01:46:36.135576 2929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:46:36.140599 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:46:36.145018 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 2929
I20260811 01:46:36.145345 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:46:36.146525 22361 external_mini_cluster.cc:1428] Setting key 21c0d18f60596a1e18671950fe963343
I20260811 01:46:36.158288 2999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d: Bootstrap starting.
I20260811 01:46:36.162905 2999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d: Neither blocks nor log segments found. Creating new log.
I20260811 01:46:36.164523 2999 log.cc:824] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d: Log is configured to *not* fsync() on all Append() calls
Aug 11 01:46:36 dist-test-slave-9gcw krb5kdc[2430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:46:36.169715 2999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d: No bootstrap required, opened a new log
I20260811 01:46:36.176384 3002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:36.149675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33272 (local address 127.21.214.126:42791)
0811 01:46:36.150986 (+ 1311us) server_negotiation.cc:206] Beginning negotiation
0811 01:46:36.151001 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 01:46:36.151052 (+ 51us) server_negotiation.cc:407] Connection header received
0811 01:46:36.153743 (+ 2691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:46:36.153776 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:46:36.154270 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:46:36.154767 (+ 497us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:46:36.156548 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:36.158540 (+ 1992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:36.160082 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:36.161221 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:36.161457 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:36.166826 (+ 5369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:46:36.166835 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:46:36.166852 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:46:36.166890 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:46:36.171852 (+ 4962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:46:36.172614 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:46:36.172621 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:46:36.172628 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:46:36.172815 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:46:36.173246 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:46:36.173253 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:46:36.173259 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:46:36.173832 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:46:36.174165 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0811 01:46:36.174704 (+ 539us) server_negotiation.cc:299] Negotiation successful
0811 01:46:36.175167 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1038,"thread_start_us":685,"threads_started":1}
I20260811 01:46:36.181697 2999 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3969bd8cd164a9486d9e8121124c37d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42791 } }
I20260811 01:46:36.182379 2999 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:46:36.182539 2999 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3969bd8cd164a9486d9e8121124c37d, State: Initialized, Role: FOLLOWER
I20260811 01:46:36.183357 2999 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3969bd8cd164a9486d9e8121124c37d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42791 } }
I20260811 01:46:36.183632 2999 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:46:36.183801 2999 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:46:36.183977 2999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:46:36.187505 2999 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3969bd8cd164a9486d9e8121124c37d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42791 } }
I20260811 01:46:36.188135 2999 leader_election.cc:302] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3969bd8cd164a9486d9e8121124c37d; no voters:
I20260811 01:46:36.189246 2999 leader_election.cc:288] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:46:36.189487 3004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:46:36.190891 3004 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d [term 1 LEADER]: Becoming Leader. State: Replica: b3969bd8cd164a9486d9e8121124c37d, State: Running, Role: LEADER
I20260811 01:46:36.191751 3004 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3969bd8cd164a9486d9e8121124c37d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42791 } }
I20260811 01:46:36.192339 2999 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:46:36.196100 3007 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3969bd8cd164a9486d9e8121124c37d. Latest consensus state: current_term: 1 leader_uuid: "b3969bd8cd164a9486d9e8121124c37d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3969bd8cd164a9486d9e8121124c37d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42791 } } }
I20260811 01:46:36.197006 3007 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d [sys.catalog]: This master's current role is: LEADER
I20260811 01:46:36.196362 3006 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3969bd8cd164a9486d9e8121124c37d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3969bd8cd164a9486d9e8121124c37d" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42791 } } }
I20260811 01:46:36.197661 3006 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d [sys.catalog]: This master's current role is: LEADER
I20260811 01:46:36.198899 3013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:46:36.209414 3013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:46:36.216480 2999 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 01:46:36.237246 3013 catalog_manager.cc:1347] Generated new cluster ID: 3e793c61ef294169af460366d8809874
I20260811 01:46:36.237489 3013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:46:36.260862 3013 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:46:36.262928 3013 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:46:36.273114 3013 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d: Generated new TSK 0
I20260811 01:46:36.274289 3013 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
I20260811 01:46:37.245179 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.214.84:36281/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42791
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:33885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 01:46:37 dist-test-slave-9gcw krb5kdc[2430](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786412797, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 01:46:37.925804 3047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:46:37.926484 3047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:46:37.926864 3047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:46:37.942703 3047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:46:37.942981 3047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:46:37.943267 3047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:46:37.964804 3047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:33885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.214.84:36281/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:46:37.967132 3047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:46:37.969843 3047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:46:37.996711 3058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:46:37.997424 3057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:46:38.007390 3047 server_base.cc:1034] running on GCE node
W20260811 01:46:38.008105 3060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:46:38.009299 3047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:46:38.012827 3047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:46:38.014223 3047 hybrid_clock.cc:648] HybridClock initialized: now 1786412798014107 us; error 76 us; skew 500 ppm
I20260811 01:46:38.014739 3047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:46:38 dist-test-slave-9gcw krb5kdc[2430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412798, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:46:38.025301 3047 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:46:38.033481 3047 webserver.cc:460] Webserver started at http://127.21.214.65:37315/ using document root <none> and password file <none>
I20260811 01:46:38.034902 3047 fs_manager.cc:359] Metadata directory not provided
I20260811 01:46:38.035156 3047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:46:38.035609 3047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 01:46:38 dist-test-slave-9gcw krb5kdc[2430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412798, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
I20260811 01:46:38.066346 3047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "7dd33a0ff7ea48dfa5490a1104dbca8d"
format_stamp: "Formatted at 2026-08-11 01:46:38 on dist-test-slave-9gcw"
server_key: "76420fd62147f58e214e67f5bcd019f2"
server_key_iv: "7a61597930fd88713c0d936e3a0e2458"
server_key_version: "kuduclusterkey@0"
I20260811 01:46:38.067781 3047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "7dd33a0ff7ea48dfa5490a1104dbca8d"
format_stamp: "Formatted at 2026-08-11 01:46:38 on dist-test-slave-9gcw"
server_key: "76420fd62147f58e214e67f5bcd019f2"
server_key_iv: "7a61597930fd88713c0d936e3a0e2458"
server_key_version: "kuduclusterkey@0"
Aug 11 01:46:38 dist-test-slave-9gcw krb5kdc[2430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412797, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
I20260811 01:46:38.092068 3047 fs_manager.cc:691] Time spent creating directory manager: real 0.023s user 0.016s sys 0.004s
I20260811 01:46:38.126053 3067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:46:38.128638 3047 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.006s sys 0.009s
I20260811 01:46:38.128923 3047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "7dd33a0ff7ea48dfa5490a1104dbca8d"
format_stamp: "Formatted at 2026-08-11 01:46:38 on dist-test-slave-9gcw"
server_key: "76420fd62147f58e214e67f5bcd019f2"
server_key_iv: "7a61597930fd88713c0d936e3a0e2458"
server_key_version: "kuduclusterkey@0"
I20260811 01:46:38.129443 3047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:46:38.237838 3047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:46:38.245189 3047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:46:38.245661 3047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:46:38.253561 3047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:46:38.274874 3047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:46:38.275096 3047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.001s
I20260811 01:46:38.275389 3047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:46:38.275550 3047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 01:46:38 dist-test-slave-9gcw krb5kdc[2430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412797, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
I20260811 01:46:38.989930 3047 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:39673
I20260811 01:46:38.990200 3208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:39673 every 8 connection(s)
I20260811 01:46:38.992097 3047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:46:39.001034 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 3047
I20260811 01:46:39.001386 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:46:39.002748 22361 external_mini_cluster.cc:1428] Setting key 5c6825fc0b6ddfa40b644ddf96fa33d8
Aug 11 01:46:39 dist-test-slave-9gcw krb5kdc[2430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412798, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:46:39.057439 3212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:38.997562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:36593 (local address 127.21.214.126:42791)
0811 01:46:39.018836 (+ 21274us) server_negotiation.cc:206] Beginning negotiation
0811 01:46:39.018846 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:46:39.021210 (+ 2364us) server_negotiation.cc:407] Connection header received
0811 01:46:39.027431 (+ 6221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:46:39.027439 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:46:39.027560 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:46:39.027777 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:46:39.031044 (+ 3267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:39.032815 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:39.036359 (+ 3544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:39.037469 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:39.037624 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:39.044559 (+ 6935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:46:39.044570 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:46:39.044576 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:46:39.044627 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:46:39.050874 (+ 6247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:46:39.052846 (+ 1972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:46:39.052865 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:46:39.052871 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:46:39.053231 (+ 360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:46:39.054394 (+ 1163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:46:39.054410 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:46:39.054414 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:46:39.054999 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:46:39.055156 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 01:46:39.056578 (+ 1422us) server_negotiation.cc:299] Negotiation successful
0811 01:46:39.056942 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21050,"thread_start_us":21290,"threads_started":1}
WARNING: no policy specified for kudu/127.21.214.66@KRBTEST.COM; defaulting to no policy
I20260811 01:46:39.060588 3211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:38.996586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:42791 (local address 127.21.214.65:36593)
0811 01:46:39.020333 (+ 23747us) negotiation.cc:107] Waiting for socket to connect
0811 01:46:39.020416 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 01:46:39.024711 (+ 4295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:46:39.028471 (+ 3760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:46:39.028486 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:46:39.029526 (+ 1040us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:46:39.030704 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:46:39.030740 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:46:39.033205 (+ 2465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:46:39.033211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:46:39.036084 (+ 2873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:46:39.036110 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:46:39.038407 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:46:39.038418 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:46:39.038796 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:39.039929 (+ 1133us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:46:39.040009 (+ 80us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:46:39.044006 (+ 3997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:46:39.051169 (+ 7163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:46:39.051183 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:46:39.051212 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:46:39.052537 (+ 1325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:46:39.053531 (+ 994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:46:39.053600 (+ 69us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:46:39.053606 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:46:39.054101 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:46:39.055284 (+ 1183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:46:39.055294 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:46:39.056008 (+ 714us) client_negotiation.cc:769] Sending connection context
0811 01:46:39.058104 (+ 2096us) client_negotiation.cc:240] Negotiation successful
0811 01:46:39.058783 (+ 679us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23260,"thread_start_us":28475,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.66@KRBTEST.COM" created.
I20260811 01:46:39.063715 3209 heartbeater.cc:347] Connected to a master server at 127.21.214.126:42791
I20260811 01:46:39.064205 3209 heartbeater.cc:464] Registering TS with master...
I20260811 01:46:39.065621 3209 heartbeater.cc:511] Master 127.21.214.126:42791 requested a full tablet report, sending...
I20260811 01:46:39.070413 2963 ts_manager.cc:194] Registered new tserver with Master: 7dd33a0ff7ea48dfa5490a1104dbca8d (127.21.214.65:39673)
I20260811 01:46:39.072861 2963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:36593
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
W20260811 01:46:39.113204 22361 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.21.214.66: real 0.105s user 0.000s sys 0.028s
I20260811 01:46:39.114280 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.214.84:36281/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42791
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:33885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:46:39.711455 3215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:46:39.711884 3215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:46:39.712061 3215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:46:39.721619 3215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:46:39.721755 3215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:46:39.721956 3215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:46:39.734443 3215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:33885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.214.84:36281/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:46:39.735953 3215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:46:39.737502 3215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:46:39.757827 3222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:46:39.758611 3223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:46:39.758913 3225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:46:39.759753 3215 server_base.cc:1034] running on GCE node
I20260811 01:46:39.760577 3215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:46:39.761898 3215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:46:39.763284 3215 hybrid_clock.cc:648] HybridClock initialized: now 1786412799763117 us; error 61 us; skew 500 ppm
I20260811 01:46:39.763773 3215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:46:39 dist-test-slave-9gcw krb5kdc[2430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412799, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:46:39.768816 3215 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:46:39.771490 3215 webserver.cc:460] Webserver started at http://127.21.214.66:44443/ using document root <none> and password file <none>
I20260811 01:46:39.772576 3215 fs_manager.cc:359] Metadata directory not provided
I20260811 01:46:39.772856 3215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:46:39.773404 3215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 01:46:39 dist-test-slave-9gcw krb5kdc[2430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412799, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
I20260811 01:46:39.793243 3215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "0cd913e1cd3447a5aeec067d4a8c25cd"
format_stamp: "Formatted at 2026-08-11 01:46:39 on dist-test-slave-9gcw"
server_key: "210c2dc8a1abdb91e949883058a2dae0"
server_key_iv: "7ea61e00f65972737832cf08b3336354"
server_key_version: "kuduclusterkey@0"
I20260811 01:46:39.794519 3215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "0cd913e1cd3447a5aeec067d4a8c25cd"
format_stamp: "Formatted at 2026-08-11 01:46:39 on dist-test-slave-9gcw"
server_key: "210c2dc8a1abdb91e949883058a2dae0"
server_key_iv: "7ea61e00f65972737832cf08b3336354"
server_key_version: "kuduclusterkey@0"
I20260811 01:46:39.803164 3215 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s
I20260811 01:46:39.821398 3234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:46:39.823621 3215 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260811 01:46:39.823848 3215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "0cd913e1cd3447a5aeec067d4a8c25cd"
format_stamp: "Formatted at 2026-08-11 01:46:39 on dist-test-slave-9gcw"
server_key: "210c2dc8a1abdb91e949883058a2dae0"
server_key_iv: "7ea61e00f65972737832cf08b3336354"
server_key_version: "kuduclusterkey@0"
I20260811 01:46:39.824131 3215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:46:39.862282 3215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:46:39.868790 3215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:46:39.869406 3215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:46:39.871266 3215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:46:39.874308 3215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:46:39.874433 3215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:46:39.874691 3215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:46:39.874861 3215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:46:39.954487 3215 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:33417
I20260811 01:46:39.954504 3366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:33417 every 8 connection(s)
I20260811 01:46:39.956262 3215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:46:39.958511 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 3215
I20260811 01:46:39.958783 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:46:39.959348 22361 external_mini_cluster.cc:1428] Setting key 0b2607e28b81f1bbc363a21a7288f0ca
Aug 11 01:46:39 dist-test-slave-9gcw krb5kdc[2430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412799, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:46:39.984817 3369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:39.960020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:56759 (local address 127.21.214.126:42791)
0811 01:46:39.960635 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0811 01:46:39.960641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:46:39.962361 (+ 1720us) server_negotiation.cc:407] Connection header received
0811 01:46:39.965116 (+ 2755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:46:39.965122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:46:39.965235 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:46:39.965365 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:46:39.967381 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:39.968362 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:39.971008 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:39.971613 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:39.971850 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:39.976680 (+ 4830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:46:39.976686 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:46:39.976690 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:46:39.976759 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:46:39.980622 (+ 3863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:46:39.981640 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:46:39.981645 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:46:39.981647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:46:39.981925 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:46:39.982453 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:46:39.982457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:46:39.982460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:46:39.982837 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:46:39.983055 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 01:46:39.983977 (+ 922us) server_negotiation.cc:299] Negotiation successful
0811 01:46:39.984229 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":361,"threads_started":1}
I20260811 01:46:39.986080 3370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:39.961019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:42791 (local address 127.21.214.66:56759)
0811 01:46:39.962189 (+ 1170us) negotiation.cc:107] Waiting for socket to connect
0811 01:46:39.962236 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 01:46:39.964808 (+ 2572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:46:39.965674 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:46:39.965683 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:46:39.966370 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:46:39.967186 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:46:39.967207 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:46:39.968571 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:46:39.968575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:46:39.970812 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:46:39.970826 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:46:39.972005 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:46:39.972026 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:46:39.972350 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:39.973322 (+ 972us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:46:39.973347 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:46:39.976291 (+ 2944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:46:39.980800 (+ 4509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:46:39.980811 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:46:39.980832 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:46:39.981481 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:46:39.982053 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:46:39.982058 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:46:39.982062 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:46:39.982307 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:46:39.982982 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:46:39.982991 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:46:39.983436 (+ 445us) client_negotiation.cc:769] Sending connection context
0811 01:46:39.983725 (+ 289us) client_negotiation.cc:240] Negotiation successful
0811 01:46:39.984570 (+ 845us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":918,"thread_start_us":538,"threads_started":1}
I20260811 01:46:39.988121 3367 heartbeater.cc:347] Connected to a master server at 127.21.214.126:42791
I20260811 01:46:39.988696 3367 heartbeater.cc:464] Registering TS with master...
I20260811 01:46:39.990052 3367 heartbeater.cc:511] Master 127.21.214.126:42791 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.214.67@KRBTEST.COM; defaulting to no policy
I20260811 01:46:39.992241 2963 ts_manager.cc:194] Registered new tserver with Master: 0cd913e1cd3447a5aeec067d4a8c25cd (127.21.214.66:33417)
I20260811 01:46:39.993669 2963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:56759
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
I20260811 01:46:40.033803 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.214.84:36281/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42791
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:33885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:46:40.078080 3209 heartbeater.cc:503] Master 127.21.214.126:42791 was elected leader, sending a full tablet report...
W20260811 01:46:40.604159 3373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:46:40.604537 3373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:46:40.604684 3373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:46:40.614308 3373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:46:40.614519 3373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:46:40.614755 3373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:46:40.627717 3373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:33885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.214.84:36281/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:46:40.629235 3373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:46:40.630782 3373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:46:40.652199 3379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:46:40.652284 3378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:46:40.652318 3381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:46:40.654152 3373 server_base.cc:1034] running on GCE node
I20260811 01:46:40.654984 3373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:46:40.656258 3373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:46:40.657516 3373 hybrid_clock.cc:648] HybridClock initialized: now 1786412800657450 us; error 40 us; skew 500 ppm
I20260811 01:46:40.657886 3373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:46:40 dist-test-slave-9gcw krb5kdc[2430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412800, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:46:40.662314 3373 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:46:40.664155 3373 webserver.cc:460] Webserver started at http://127.21.214.67:39977/ using document root <none> and password file <none>
I20260811 01:46:40.665205 3373 fs_manager.cc:359] Metadata directory not provided
I20260811 01:46:40.665524 3373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:46:40.666060 3373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 01:46:40 dist-test-slave-9gcw krb5kdc[2430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412800, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
I20260811 01:46:40.685456 3373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "889c114716c74c70a4770b1aaa11d764"
format_stamp: "Formatted at 2026-08-11 01:46:40 on dist-test-slave-9gcw"
server_key: "7d7f7a76ffb5c2e448a0c3a7e030a6cb"
server_key_iv: "b28d920f63bc72a8a7d28f76be3f587c"
server_key_version: "kuduclusterkey@0"
I20260811 01:46:40.686648 3373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "889c114716c74c70a4770b1aaa11d764"
format_stamp: "Formatted at 2026-08-11 01:46:40 on dist-test-slave-9gcw"
server_key: "7d7f7a76ffb5c2e448a0c3a7e030a6cb"
server_key_iv: "b28d920f63bc72a8a7d28f76be3f587c"
server_key_version: "kuduclusterkey@0"
I20260811 01:46:40.694275 3373 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260811 01:46:40.712949 3389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:46:40.714922 3373 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 01:46:40.715149 3373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "889c114716c74c70a4770b1aaa11d764"
format_stamp: "Formatted at 2026-08-11 01:46:40 on dist-test-slave-9gcw"
server_key: "7d7f7a76ffb5c2e448a0c3a7e030a6cb"
server_key_iv: "b28d920f63bc72a8a7d28f76be3f587c"
server_key_version: "kuduclusterkey@0"
I20260811 01:46:40.715459 3373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:46:40.758100 3373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:46:40.764143 3373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:46:40.764717 3373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:46:40.766098 3373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:46:40.767994 3373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:46:40.768090 3373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:46:40.768303 3373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:46:40.768394 3373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:46:40.830374 3373 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:37357
I20260811 01:46:40.830411 3511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:37357 every 8 connection(s)
I20260811 01:46:40.832152 3373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:46:40.842017 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 3373
I20260811 01:46:40.842219 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:46:40.842705 22361 external_mini_cluster.cc:1428] Setting key 5755505cd59fe8ce628ae98dca1a8ce1
Aug 11 01:46:40 dist-test-slave-9gcw krb5kdc[2430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412800, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:46:40.859334 3514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:40.835550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:60045 (local address 127.21.214.126:42791)
0811 01:46:40.836249 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0811 01:46:40.836254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:46:40.837830 (+ 1576us) server_negotiation.cc:407] Connection header received
0811 01:46:40.840080 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:46:40.840087 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:46:40.840210 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:46:40.840368 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:46:40.842245 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:40.843198 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:40.845343 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:40.845891 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:40.846023 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:40.850577 (+ 4554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:46:40.850583 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:46:40.850587 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:46:40.850631 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:46:40.854574 (+ 3943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:46:40.855742 (+ 1168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:46:40.855746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:46:40.855749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:46:40.855897 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:46:40.856623 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:46:40.856626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:46:40.856628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:46:40.857049 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:46:40.857318 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 01:46:40.858528 (+ 1210us) server_negotiation.cc:299] Negotiation successful
0811 01:46:40.858834 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":453,"threads_started":1}
I20260811 01:46:40.860970 3515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:40.836339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:42791 (local address 127.21.214.67:60045)
0811 01:46:40.837631 (+ 1292us) negotiation.cc:107] Waiting for socket to connect
0811 01:46:40.837682 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 01:46:40.839786 (+ 2104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:46:40.840688 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:46:40.840698 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:46:40.841350 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:46:40.842049 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:46:40.842069 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:46:40.843363 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:46:40.843373 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:46:40.845187 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:46:40.845210 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:46:40.846225 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:46:40.846235 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:46:40.846514 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:40.847239 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:46:40.847265 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:46:40.850228 (+ 2963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:46:40.854794 (+ 4566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:46:40.854806 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:46:40.854825 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:46:40.855594 (+ 769us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:46:40.856059 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:46:40.856063 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:46:40.856066 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:46:40.856473 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:46:40.857261 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:46:40.857271 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:46:40.857968 (+ 697us) client_negotiation.cc:769] Sending connection context
0811 01:46:40.858386 (+ 418us) client_negotiation.cc:240] Negotiation successful
0811 01:46:40.859134 (+ 748us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":987,"thread_start_us":648,"threads_started":1}
I20260811 01:46:40.862807 3512 heartbeater.cc:347] Connected to a master server at 127.21.214.126:42791
I20260811 01:46:40.863188 3512 heartbeater.cc:464] Registering TS with master...
I20260811 01:46:40.864037 3512 heartbeater.cc:511] Master 127.21.214.126:42791 requested a full tablet report, sending...
I20260811 01:46:40.865897 2963 ts_manager.cc:194] Registered new tserver with Master: 889c114716c74c70a4770b1aaa11d764 (127.21.214.67:37357)
I20260811 01:46:40.866945 2963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:60045
I20260811 01:46:40.872287 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:46:40.895009 3514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:40.878283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33288 (local address 127.21.214.126:42791)
0811 01:46:40.878566 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 01:46:40.878576 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:46:40.879089 (+ 513us) server_negotiation.cc:407] Connection header received
0811 01:46:40.879273 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:46:40.879278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:46:40.879387 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:46:40.879488 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:46:40.881452 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:40.882368 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:40.884098 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:40.884596 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:40.884702 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:40.888254 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:46:40.888260 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:46:40.888263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:46:40.888300 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:46:40.891985 (+ 3685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:46:40.892933 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:46:40.892937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:46:40.892940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:46:40.893094 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:46:40.893662 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:46:40.893665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:46:40.893667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:46:40.893992 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:46:40.894210 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 01:46:40.894472 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 01:46:40.894673 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 01:46:40.899941 2963 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:46:40.910382 2963 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:46:40.946144 3526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:40.929426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:33417 (local address 127.0.0.1:37754)
0811 01:46:40.936505 (+ 7079us) negotiation.cc:107] Waiting for socket to connect
0811 01:46:40.936570 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 01:46:40.936892 (+ 322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:46:40.937935 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:46:40.937943 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:46:40.937973 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:46:40.938490 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:46:40.938501 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:46:40.940163 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:46:40.940170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:46:40.943220 (+ 3050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:46:40.943235 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:46:40.944829 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:46:40.944838 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:46:40.945125 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:40.945231 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 01:46:40.945404 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 01:46:40.945608 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3316,"thread_start_us":766,"threads_started":1}
I20260811 01:46:40.947165 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:40.930114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37754 (local address 127.21.214.66:33417)
0811 01:46:40.931209 (+ 1095us) server_negotiation.cc:206] Beginning negotiation
0811 01:46:40.931220 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:46:40.937135 (+ 5915us) server_negotiation.cc:407] Connection header received
0811 01:46:40.937308 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:46:40.937316 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:46:40.937701 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:46:40.937929 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:46:40.938770 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:40.940004 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:40.943408 (+ 3404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:40.944476 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:40.944644 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:40.944978 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0811 01:46:40.946430 (+ 1452us) server_negotiation.cc:299] Negotiation successful
0811 01:46:40.946660 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":872,"thread_start_us":646,"threads_started":1}
I20260811 01:46:40.952718 3291 tablet_service.cc:1467] Processing CreateTablet for tablet 07959f4295444f2ebb71506827e7a80c (DEFAULT_TABLE table=test-table [id=52747f2093434a4e8f1cfe401e4b290c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:46:40.954982 3291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07959f4295444f2ebb71506827e7a80c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:46:40.957636 3531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:40.931451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:37357 (local address 127.0.0.1:37242)
0811 01:46:40.946011 (+ 14560us) negotiation.cc:107] Waiting for socket to connect
0811 01:46:40.946035 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 01:46:40.946290 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:46:40.947609 (+ 1319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:46:40.947614 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:46:40.947654 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:46:40.948328 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:46:40.948342 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:46:40.950654 (+ 2312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:46:40.950661 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:46:40.953846 (+ 3185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:46:40.953867 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:46:40.956518 (+ 2651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:46:40.956525 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:46:40.956727 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:40.956763 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 01:46:40.956854 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 01:46:40.957035 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14410,"mutex_wait_us":1711,"thread_start_us":11654,"threads_started":1}
I20260811 01:46:40.958563 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:40.931886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60432 (local address 127.21.214.65:39673)
0811 01:46:40.933111 (+ 1225us) server_negotiation.cc:206] Beginning negotiation
0811 01:46:40.933124 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:46:40.944781 (+ 11657us) server_negotiation.cc:407] Connection header received
0811 01:46:40.945469 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:46:40.945478 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:46:40.945826 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:46:40.946102 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:46:40.947580 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:40.949585 (+ 2005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:40.954699 (+ 5114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:40.956503 (+ 1804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:40.956643 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:40.956878 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 01:46:40.957823 (+ 945us) server_negotiation.cc:299] Negotiation successful
0811 01:46:40.958053 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":909,"thread_start_us":604,"threads_started":1}
I20260811 01:46:40.958595 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:40.931912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37242 (local address 127.21.214.67:37357)
0811 01:46:40.934019 (+ 2107us) server_negotiation.cc:206] Beginning negotiation
0811 01:46:40.934025 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:46:40.946167 (+ 12142us) server_negotiation.cc:407] Connection header received
0811 01:46:40.946532 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:46:40.946542 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:46:40.946853 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:46:40.947069 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:46:40.948568 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:40.950491 (+ 1923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:40.954269 (+ 3778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:40.955940 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:40.956089 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:40.956343 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 01:46:40.957907 (+ 1564us) server_negotiation.cc:299] Negotiation successful
0811 01:46:40.958116 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1942,"thread_start_us":1677,"threads_started":1}
I20260811 01:46:40.959707 3532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:40.931272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:39673 (local address 127.0.0.1:60432)
0811 01:46:40.944582 (+ 13310us) negotiation.cc:107] Waiting for socket to connect
0811 01:46:40.944611 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 01:46:40.945217 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:46:40.946598 (+ 1381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:46:40.946603 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:46:40.946629 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:46:40.947176 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:46:40.947193 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:46:40.949744 (+ 2551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:46:40.949750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:46:40.954474 (+ 4724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:46:40.954490 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:46:40.957208 (+ 2718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:46:40.957216 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:46:40.957432 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:40.957468 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 01:46:40.958877 (+ 1409us) client_negotiation.cc:240] Negotiation successful
0811 01:46:40.959057 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13109,"mutex_wait_us":1517,"thread_start_us":12870,"threads_started":1}
I20260811 01:46:40.964299 3444 tablet_service.cc:1467] Processing CreateTablet for tablet 07959f4295444f2ebb71506827e7a80c (DEFAULT_TABLE table=test-table [id=52747f2093434a4e8f1cfe401e4b290c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:46:40.965449 3125 tablet_service.cc:1467] Processing CreateTablet for tablet 07959f4295444f2ebb71506827e7a80c (DEFAULT_TABLE table=test-table [id=52747f2093434a4e8f1cfe401e4b290c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:46:40.966306 3444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07959f4295444f2ebb71506827e7a80c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:46:40.967465 3125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07959f4295444f2ebb71506827e7a80c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:46:40.977937 3533 tablet_bootstrap.cc:492] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd: Bootstrap starting.
I20260811 01:46:40.982411 3533 tablet_bootstrap.cc:654] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd: Neither blocks nor log segments found. Creating new log.
I20260811 01:46:40.984735 3533 log.cc:824] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd: Log is configured to *not* fsync() on all Append() calls
I20260811 01:46:40.987653 3537 tablet_bootstrap.cc:492] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d: Bootstrap starting.
I20260811 01:46:40.988881 3533 tablet_bootstrap.cc:492] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd: No bootstrap required, opened a new log
I20260811 01:46:40.989228 3533 ts_tablet_manager.cc:1397] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.006s
I20260811 01:46:40.989590 3536 tablet_bootstrap.cc:492] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764: Bootstrap starting.
I20260811 01:46:40.993909 3536 tablet_bootstrap.cc:654] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764: Neither blocks nor log segments found. Creating new log.
I20260811 01:46:40.994940 3537 tablet_bootstrap.cc:654] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d: Neither blocks nor log segments found. Creating new log.
I20260811 01:46:40.995800 3536 log.cc:824] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764: Log is configured to *not* fsync() on all Append() calls
I20260811 01:46:40.997308 3367 heartbeater.cc:503] Master 127.21.214.126:42791 was elected leader, sending a full tablet report...
I20260811 01:46:40.998299 3537 log.cc:824] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d: Log is configured to *not* fsync() on all Append() calls
I20260811 01:46:41.005203 3537 tablet_bootstrap.cc:492] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d: No bootstrap required, opened a new log
I20260811 01:46:41.005587 3537 ts_tablet_manager.cc:1397] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.000s
I20260811 01:46:41.005669 3536 tablet_bootstrap.cc:492] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764: No bootstrap required, opened a new log
I20260811 01:46:41.006004 3536 ts_tablet_manager.cc:1397] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.000s
I20260811 01:46:41.006683 3533 raft_consensus.cc:348] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd913e1cd3447a5aeec067d4a8c25cd" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33417 } } peers { permanent_uuid: "7dd33a0ff7ea48dfa5490a1104dbca8d" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39673 } } peers { permanent_uuid: "889c114716c74c70a4770b1aaa11d764" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 37357 } }
I20260811 01:46:41.007444 3533 raft_consensus.cc:374] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:46:41.007623 3533 raft_consensus.cc:729] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd913e1cd3447a5aeec067d4a8c25cd, State: Initialized, Role: FOLLOWER
I20260811 01:46:41.008647 3533 consensus_queue.cc:260] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd913e1cd3447a5aeec067d4a8c25cd" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33417 } } peers { permanent_uuid: "7dd33a0ff7ea48dfa5490a1104dbca8d" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39673 } } peers { permanent_uuid: "889c114716c74c70a4770b1aaa11d764" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 37357 } }
I20260811 01:46:41.011821 3533 ts_tablet_manager.cc:1428] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd: Time spent starting tablet: real 0.022s user 0.018s sys 0.001s
I20260811 01:46:41.018087 3537 raft_consensus.cc:348] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd913e1cd3447a5aeec067d4a8c25cd" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33417 } } peers { permanent_uuid: "7dd33a0ff7ea48dfa5490a1104dbca8d" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39673 } } peers { permanent_uuid: "889c114716c74c70a4770b1aaa11d764" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 37357 } }
I20260811 01:46:41.018507 3537 raft_consensus.cc:374] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:46:41.018672 3537 raft_consensus.cc:729] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd33a0ff7ea48dfa5490a1104dbca8d, State: Initialized, Role: FOLLOWER
I20260811 01:46:41.019475 3537 consensus_queue.cc:260] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd913e1cd3447a5aeec067d4a8c25cd" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33417 } } peers { permanent_uuid: "7dd33a0ff7ea48dfa5490a1104dbca8d" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39673 } } peers { permanent_uuid: "889c114716c74c70a4770b1aaa11d764" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 37357 } }
I20260811 01:46:41.021210 3536 raft_consensus.cc:348] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd913e1cd3447a5aeec067d4a8c25cd" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33417 } } peers { permanent_uuid: "7dd33a0ff7ea48dfa5490a1104dbca8d" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39673 } } peers { permanent_uuid: "889c114716c74c70a4770b1aaa11d764" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 37357 } }
I20260811 01:46:41.021852 3537 ts_tablet_manager.cc:1428] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260811 01:46:41.021999 3536 raft_consensus.cc:374] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:46:41.022217 3536 raft_consensus.cc:729] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 889c114716c74c70a4770b1aaa11d764, State: Initialized, Role: FOLLOWER
I20260811 01:46:41.023049 3536 consensus_queue.cc:260] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd913e1cd3447a5aeec067d4a8c25cd" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33417 } } peers { permanent_uuid: "7dd33a0ff7ea48dfa5490a1104dbca8d" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39673 } } peers { permanent_uuid: "889c114716c74c70a4770b1aaa11d764" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 37357 } }
I20260811 01:46:41.023813 3540 raft_consensus.cc:482] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:46:41.025408 3512 heartbeater.cc:503] Master 127.21.214.126:42791 was elected leader, sending a full tablet report...
I20260811 01:46:41.026432 3536 ts_tablet_manager.cc:1428] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764: Time spent starting tablet: real 0.020s user 0.009s sys 0.010s
I20260811 01:46:41.024345 3540 raft_consensus.cc:504] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd913e1cd3447a5aeec067d4a8c25cd" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33417 } } peers { permanent_uuid: "7dd33a0ff7ea48dfa5490a1104dbca8d" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39673 } } peers { permanent_uuid: "889c114716c74c70a4770b1aaa11d764" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 37357 } }
W20260811 01:46:41.029193 3210 tablet.cc:2369] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:46:41.031250 3540 leader_election.cc:288] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd33a0ff7ea48dfa5490a1104dbca8d (127.21.214.65:39673), 889c114716c74c70a4770b1aaa11d764 (127.21.214.67:37357)
I20260811 01:46:41.039784 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:41.030217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:44521 (local address 127.21.214.65:39673)
0811 01:46:41.030509 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 01:46:41.030515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:46:41.031735 (+ 1220us) server_negotiation.cc:407] Connection header received
0811 01:46:41.032158 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:46:41.032176 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:46:41.032287 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:46:41.032420 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:46:41.033295 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:41.034469 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:41.037261 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:41.038265 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:41.038501 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:41.038690 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 01:46:41.039313 (+ 623us) server_negotiation.cc:299] Negotiation successful
0811 01:46:41.039464 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 01:46:41.039932 3545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:41.029552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:39673 (local address 127.21.214.66:44521)
0811 01:46:41.031561 (+ 2009us) negotiation.cc:107] Waiting for socket to connect
0811 01:46:41.031591 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 01:46:41.031933 (+ 342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:46:41.032443 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:46:41.032448 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:46:41.032479 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:46:41.033037 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:46:41.033050 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:46:41.034622 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:46:41.034628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:46:41.037004 (+ 2376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:46:41.037023 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:46:41.038687 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:46:41.038695 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:46:41.039048 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:41.039076 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 01:46:41.039281 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 01:46:41.039481 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1463,"threads_started":1}
I20260811 01:46:41.043558 3148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07959f4295444f2ebb71506827e7a80c" candidate_uuid: "0cd913e1cd3447a5aeec067d4a8c25cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd33a0ff7ea48dfa5490a1104dbca8d" is_pre_election: true
I20260811 01:46:41.044528 3148 raft_consensus.cc:2454] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cd913e1cd3447a5aeec067d4a8c25cd in term 0.
I20260811 01:46:41.045614 3235 leader_election.cc:302] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cd913e1cd3447a5aeec067d4a8c25cd, 7dd33a0ff7ea48dfa5490a1104dbca8d; no voters:
I20260811 01:46:41.046353 3540 raft_consensus.cc:2781] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:46:41.046581 3540 raft_consensus.cc:482] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:46:41.046718 3540 raft_consensus.cc:3037] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:46:41.046914 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:41.030161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:43407 (local address 127.21.214.67:37357)
0811 01:46:41.030417 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 01:46:41.030451 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0811 01:46:41.040300 (+ 9849us) server_negotiation.cc:407] Connection header received
0811 01:46:41.040501 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:46:41.040513 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:46:41.040581 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:46:41.040713 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:46:41.041178 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:41.042157 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:41.044180 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:41.045569 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:41.045791 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:41.045870 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 01:46:41.046388 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 01:46:41.046559 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 01:46:41.047076 3545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:41.030128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:37357 (local address 127.21.214.66:43407)
0811 01:46:41.040181 (+ 10053us) negotiation.cc:107] Waiting for socket to connect
0811 01:46:41.040201 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 01:46:41.040357 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:46:41.040731 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:46:41.040736 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:46:41.040763 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:46:41.040989 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:46:41.040999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:46:41.042307 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:46:41.042312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:46:41.043978 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:46:41.043994 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:46:41.045810 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:46:41.045818 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:46:41.046165 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:41.046218 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 01:46:41.046340 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 01:46:41.046553 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9907,"thread_start_us":760,"threads_started":1}
I20260811 01:46:41.048488 3467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07959f4295444f2ebb71506827e7a80c" candidate_uuid: "0cd913e1cd3447a5aeec067d4a8c25cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "889c114716c74c70a4770b1aaa11d764" is_pre_election: true
I20260811 01:46:41.049207 3467 raft_consensus.cc:2454] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cd913e1cd3447a5aeec067d4a8c25cd in term 0.
I20260811 01:46:41.050829 3540 raft_consensus.cc:504] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd913e1cd3447a5aeec067d4a8c25cd" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33417 } } peers { permanent_uuid: "7dd33a0ff7ea48dfa5490a1104dbca8d" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39673 } } peers { permanent_uuid: "889c114716c74c70a4770b1aaa11d764" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 37357 } }
I20260811 01:46:41.051632 3540 leader_election.cc:288] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [CANDIDATE]: Term 1 election: Requested vote from peers 7dd33a0ff7ea48dfa5490a1104dbca8d (127.21.214.65:39673), 889c114716c74c70a4770b1aaa11d764 (127.21.214.67:37357)
I20260811 01:46:41.052167 3148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07959f4295444f2ebb71506827e7a80c" candidate_uuid: "0cd913e1cd3447a5aeec067d4a8c25cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd33a0ff7ea48dfa5490a1104dbca8d"
I20260811 01:46:41.052181 3467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07959f4295444f2ebb71506827e7a80c" candidate_uuid: "0cd913e1cd3447a5aeec067d4a8c25cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "889c114716c74c70a4770b1aaa11d764"
I20260811 01:46:41.052424 3148 raft_consensus.cc:3037] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:46:41.052496 3467 raft_consensus.cc:3037] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:46:41.055627 3467 raft_consensus.cc:2454] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cd913e1cd3447a5aeec067d4a8c25cd in term 1.
I20260811 01:46:41.055656 3148 raft_consensus.cc:2454] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cd913e1cd3447a5aeec067d4a8c25cd in term 1.
I20260811 01:46:41.056339 3238 leader_election.cc:302] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cd913e1cd3447a5aeec067d4a8c25cd, 889c114716c74c70a4770b1aaa11d764; no voters:
I20260811 01:46:41.056985 3540 raft_consensus.cc:2781] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:46:41.057732 3540 raft_consensus.cc:686] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [term 1 LEADER]: Becoming Leader. State: Replica: 0cd913e1cd3447a5aeec067d4a8c25cd, State: Running, Role: LEADER
I20260811 01:46:41.058465 3540 consensus_queue.cc:237] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd913e1cd3447a5aeec067d4a8c25cd" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33417 } } peers { permanent_uuid: "7dd33a0ff7ea48dfa5490a1104dbca8d" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39673 } } peers { permanent_uuid: "889c114716c74c70a4770b1aaa11d764" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 37357 } }
I20260811 01:46:41.065243 2962 catalog_manager.cc:5495] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cd913e1cd3447a5aeec067d4a8c25cd (127.21.214.66). New cstate: current_term: 1 leader_uuid: "0cd913e1cd3447a5aeec067d4a8c25cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd913e1cd3447a5aeec067d4a8c25cd" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 33417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd33a0ff7ea48dfa5490a1104dbca8d" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "889c114716c74c70a4770b1aaa11d764" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 37357 } health_report { overall_health: UNKNOWN } } }
W20260811 01:46:41.084506 3513 tablet.cc:2369] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:46:41.104596 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:41.098187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37758 (local address 127.21.214.66:33417)
0811 01:46:41.098413 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 01:46:41.098419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:46:41.098441 (+ 22us) server_negotiation.cc:407] Connection header received
0811 01:46:41.098628 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:46:41.098632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:46:41.098727 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:46:41.098905 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:46:41.099331 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:41.100135 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:41.101787 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:41.102330 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:41.102485 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:41.103258 (+ 773us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:46:41.103820 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:46:41.103997 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 01:46:41.104057 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 01:46:41.104145 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 01:46:41.114389 3148 raft_consensus.cc:1261] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d [term 1 FOLLOWER]: Refusing update from remote peer 0cd913e1cd3447a5aeec067d4a8c25cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:46:41.114408 3467 raft_consensus.cc:1261] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764 [term 1 FOLLOWER]: Refusing update from remote peer 0cd913e1cd3447a5aeec067d4a8c25cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:46:41.115504 3546 consensus_queue.cc:1035] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "889c114716c74c70a4770b1aaa11d764" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 37357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:46:41.115924 3540 consensus_queue.cc:1035] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd33a0ff7ea48dfa5490a1104dbca8d" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:46:41.130733 3550 mvcc.cc:205] Tried to move back new op lower bound from 7317146833352810496 to 7317146833140854784. Current Snapshot: MvccSnapshot[applied={T|T < 7317146833352810496}]
I20260811 01:46:41.131945 3551 mvcc.cc:205] Tried to move back new op lower bound from 7317146833352810496 to 7317146833140854784. Current Snapshot: MvccSnapshot[applied={T|T < 7317146833352810496}]
I20260811 01:46:41.144398 3552 mvcc.cc:205] Tried to move back new op lower bound from 7317146833352810496 to 7317146833140854784. Current Snapshot: MvccSnapshot[applied={T|T < 7317146833352810496}]
I20260811 01:46:41.160856 2963 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:46:41.161461 2963 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:46:41.166875 2963 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b3969bd8cd164a9486d9e8121124c37d: Sending DeleteTablet for 3 replicas of tablet 07959f4295444f2ebb71506827e7a80c
I20260811 01:46:41.168867 3291 tablet_service.cc:1514] Processing DeleteTablet for tablet 07959f4295444f2ebb71506827e7a80c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:46:41 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:37754
I20260811 01:46:41.168953 3125 tablet_service.cc:1514] Processing DeleteTablet for tablet 07959f4295444f2ebb71506827e7a80c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:46:41 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:60432
I20260811 01:46:41.169858 3444 tablet_service.cc:1514] Processing DeleteTablet for tablet 07959f4295444f2ebb71506827e7a80c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:46:41 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:37242
I20260811 01:46:41.170867 3563 tablet_replica.cc:330] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd: stopping tablet replica
I20260811 01:46:41.171592 3563 raft_consensus.cc:2229] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:46:41.172776 3563 raft_consensus.cc:2258] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:46:41.175781 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 3047
I20260811 01:46:41.176285 3563 ts_tablet_manager.cc:1905] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:46:41.184005 3565 tablet_replica.cc:330] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764: stopping tablet replica
I20260811 01:46:41.184861 3565 raft_consensus.cc:2229] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:46:41.185680 3565 raft_consensus.cc:2258] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:46:41.188196 3563 ts_tablet_manager.cc:1918] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:46:41.188472 3563 log.cc:1196] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/07959f4295444f2ebb71506827e7a80c
I20260811 01:46:41.189427 3563 ts_tablet_manager.cc:1939] T 07959f4295444f2ebb71506827e7a80c P 0cd913e1cd3447a5aeec067d4a8c25cd: Deleting consensus metadata
I20260811 01:46:41.189990 3565 ts_tablet_manager.cc:1905] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:46:41.190444 3564 tablet_replica.cc:330] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d: stopping tablet replica
I20260811 01:46:41.191094 3564 raft_consensus.cc:2229] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:46:41.191596 2946 catalog_manager.cc:4841] TS 0cd913e1cd3447a5aeec067d4a8c25cd (127.21.214.66:33417): tablet 07959f4295444f2ebb71506827e7a80c (table test-table [id=52747f2093434a4e8f1cfe401e4b290c]) successfully deleted
I20260811 01:46:41.191876 3564 raft_consensus.cc:2258] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:46:41.197757 3565 ts_tablet_manager.cc:1918] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:46:41.197810 3564 ts_tablet_manager.cc:1905] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:46:41.198087 3565 log.cc:1196] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/07959f4295444f2ebb71506827e7a80c
I20260811 01:46:41.198716 3565 ts_tablet_manager.cc:1939] T 07959f4295444f2ebb71506827e7a80c P 889c114716c74c70a4770b1aaa11d764: Deleting consensus metadata
I20260811 01:46:41.200840 2949 catalog_manager.cc:4841] TS 889c114716c74c70a4770b1aaa11d764 (127.21.214.67:37357): tablet 07959f4295444f2ebb71506827e7a80c (table test-table [id=52747f2093434a4e8f1cfe401e4b290c]) successfully deleted
Aug 11 01:46:41 dist-test-slave-9gcw krb5kdc[2430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:46:41.205955 3564 ts_tablet_manager.cc:1918] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:46:41.206205 3564 log.cc:1196] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/07959f4295444f2ebb71506827e7a80c
I20260811 01:46:41.206990 3564 ts_tablet_manager.cc:1939] T 07959f4295444f2ebb71506827e7a80c P 7dd33a0ff7ea48dfa5490a1104dbca8d: Deleting consensus metadata
I20260811 01:46:41.208837 2946 catalog_manager.cc:4841] TS 7dd33a0ff7ea48dfa5490a1104dbca8d (127.21.214.65:39673): tablet 07959f4295444f2ebb71506827e7a80c (table test-table [id=52747f2093434a4e8f1cfe401e4b290c]) successfully deleted
I20260811 01:46:41.212792 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:41.177323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60434 (local address 127.21.214.65:39673)
0811 01:46:41.177613 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0811 01:46:41.177624 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:46:41.186692 (+ 9068us) server_negotiation.cc:407] Connection header received
0811 01:46:41.187017 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:46:41.187024 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:46:41.187163 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:46:41.187276 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:46:41.192354 (+ 5078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:41.193662 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:41.196076 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:41.197019 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:41.197222 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:41.204256 (+ 7034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:46:41.204275 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:46:41.204297 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:46:41.204362 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:46:41.209220 (+ 4858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:46:41.210137 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:46:41.210148 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:46:41.210151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:46:41.210340 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:46:41.210754 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:46:41.210760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:46:41.210763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:46:41.211315 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:46:41.211697 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0811 01:46:41.212062 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 01:46:41.212377 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260811 01:46:41.213724 3199 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:46:41.310088 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 3047
I20260811 01:46:41.343333 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 3215
Aug 11 01:46:41 dist-test-slave-9gcw krb5kdc[2430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.66@KRBTEST.COM
I20260811 01:46:41.362295 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:41.344276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37760 (local address 127.21.214.66:33417)
0811 01:46:41.344515 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 01:46:41.344521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:46:41.344543 (+ 22us) server_negotiation.cc:407] Connection header received
0811 01:46:41.344716 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:46:41.344721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:46:41.344806 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:46:41.344920 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:46:41.346342 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:41.347364 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:41.348697 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:41.349234 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:41.349374 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:41.354323 (+ 4949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:46:41.354342 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:46:41.354361 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:46:41.354448 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:46:41.358784 (+ 4336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:46:41.359728 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:46:41.359736 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:46:41.359738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:46:41.359897 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:46:41.360332 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:46:41.360338 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:46:41.360341 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:46:41.360771 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:46:41.361188 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0811 01:46:41.361491 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 01:46:41.361772 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 01:46:41.363296 3362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:46:41.482177 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 3215
I20260811 01:46:41.506187 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 3373
Aug 11 01:46:41 dist-test-slave-9gcw krb5kdc[2430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.67@KRBTEST.COM
I20260811 01:46:41.522979 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:46:41.506987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37250 (local address 127.21.214.67:37357)
0811 01:46:41.507187 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 01:46:41.507192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:46:41.507212 (+ 20us) server_negotiation.cc:407] Connection header received
0811 01:46:41.507385 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:46:41.507389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:46:41.507466 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:46:41.507553 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:46:41.508921 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:41.509747 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:41.511105 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:46:41.511496 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:46:41.511597 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:46:41.516495 (+ 4898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:46:41.516512 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:46:41.516546 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:46:41.516585 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:46:41.520144 (+ 3559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:46:41.520868 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:46:41.520876 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:46:41.520878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:46:41.521007 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:46:41.521416 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:46:41.521419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:46:41.521421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:46:41.521834 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:46:41.522052 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 01:46:41.522408 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 01:46:41.522581 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 01:46:41.523713 3507 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:46:41.619009 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 3373
I20260811 01:46:41.643199 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 2929
I20260811 01:46:41.644695 2994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:46:41.762559 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 2929
I20260811 01:46:41.793885 22361 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 01:46:42.140192 22361 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 01:46:42.140508 22361 mini_ranger.cc:66] Stopping Ranger...
I20260811 01:46:42.486826 22361 mini_ranger.cc:68] Stopped Ranger
2026-08-11 01:46:42.487 UTC [2463] LOG: received smart shutdown request
2026-08-11 01:46:42.491 UTC [2463] LOG: background worker "logical replication launcher" (PID 2473) exited with exit code 1
2026-08-11 01:46:42.492 UTC [2467] LOG: shutting down
2026-08-11 01:46:42.696 UTC [2463] LOG: database system is shut down
2026-08-11T01:46:42Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (55611 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:46:42 dist-test-slave-9gcw krb5kdc[3581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:46:42 dist-test-slave-9gcw krb5kdc[3581](info): set up 2 sockets
Aug 11 01:46:42 dist-test-slave-9gcw krb5kdc[3581](info): commencing operation
krb5kdc: starting...
W20260811 01:46:44.090198 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.002s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:46:44 dist-test-slave-9gcw krb5kdc[3581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:46:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:46:44Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.214.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.214.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.214.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/rangeradmin_127.21.214.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.214.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/rangeradmin_127.21.214.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.214.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.214.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.214.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/rangerlookup_127.21.214.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.214.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/rangerlookup_127.21.214.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.214.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.214.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.214.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/HTTP_127.21.214.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.214.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/HTTP_127.21.214.84@KRBTEST.COM.keytab.
I20260811 01:46:44.525115 22361 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskw71eKA/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/postgres -l logfile start
2026-08-11 01:46:47.412 UTC [3617] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 01:46:47.412 UTC [3617] LOG: listening on IPv4 address "127.21.214.84", port 54427
2026-08-11 01:46:47.417 UTC [3617] LOG: listening on Unix socket "/tmp/.s.PGSQL.54427"
2026-08-11 01:46:47.443 UTC [3620] LOG: database system was shut down at 2026-08-11 01:46:45 UTC
2026-08-11 01:46:47.447 UTC [3617] LOG: database system is ready to accept connections
I20260811 01:46:48.613673 22361 mini_postgres.cc:96] Postgres bound to 54427
2026-08-11 01:46:48.639 UTC [3628] FATAL: database "slave" does not exist
127.21.214.84:54427 - accepting connections
I20260811 01:46:48.640681 22361 mini_ranger.cc:161] Starting Ranger...
I20260811 01:46:48.676505 22361 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 01:46:49.327062 22361 mini_ranger.cc:87] Created ranger Postgres database
I20260811 01:46:49.327176 22361 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-admin
2026-08-11 01:46:49,652 [I] DB FLAVOR :POSTGRES
2026-08-11 01:46:49,652 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 01:46:49,652 [I] Checking connection..
2026-08-11 01:46:49,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 01:46:49,900 [I] Checking connection passed.
2026-08-11 01:46:49,901 [I] --------- Verifying version history table ---------
2026-08-11 01:46:49,901 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 01:46:50,152 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 01:46:50,152 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 01:46:50,428 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 01:46:50,429 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 01:46:50,429 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 01:46:50.695 UTC [3703] WARNING: there is no transaction in progress
2026-08-11 01:46:50,704 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 01:46:50,705 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 01:46:50,983 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 01:46:50,984 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 01:46:50,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 01:46:51,271 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 01:46:51,524 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9gcw.c.gcp-upstream.internal','N') ;"
2026-08-11 01:46:51,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 01:46:52,053 [I] Table x_portal_user does not exist in database ranger
2026-08-11 01:46:52,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 01:46:52,353 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 01:46:52,354 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 01:46:52,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 01:46:52,909 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 01:46:52,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 01:46:54.001 UTC [3832] WARNING: there is no transaction in progress
2026-08-11 01:46:54.020 UTC [3832] WARNING: there is no transaction in progress
2026-08-11 01:46:54.039 UTC [3832] WARNING: there is no transaction in progress
2026-08-11 01:46:54.057 UTC [3832] WARNING: there is no transaction in progress
2026-08-11 01:46:54.076 UTC [3832] WARNING: there is no transaction in progress
2026-08-11 01:46:54.102 UTC [3832] WARNING: there is no transaction in progress
2026-08-11 01:46:54.167 UTC [3832] WARNING: there is no transaction in progress
2026-08-11 01:46:54.175 UTC [3832] WARNING: there is no transaction in progress
2026-08-11 01:46:54.192 UTC [3832] WARNING: there is no transaction in progress
2026-08-11 01:46:54.204 UTC [3832] WARNING: there is no transaction in progress
2026-08-11 01:46:54.216 UTC [3832] WARNING: there is no transaction in progress
2026-08-11 01:46:54.232 UTC [3832] WARNING: there is no transaction in progress
2026-08-11 01:46:54.245 UTC [3832] WARNING: there is no transaction in progress
2026-08-11 01:46:54.253 UTC [3832] WARNING: there is no transaction in progress
2026-08-11 01:46:54.261 UTC [3832] WARNING: there is no transaction in progress
2026-08-11 01:46:54.269 UTC [3832] WARNING: there is no transaction in progress
2026-08-11 01:46:55,020 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 01:46:55,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 01:46:55,288 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 01:46:55,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 01:46:55,543 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 01:46:55,543 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 01:46:55,827 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 01:46:55,827 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 01:46:56,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 01:46:56,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9gcw.c.gcp-upstream.internal';"
2026-08-11 01:46:56,630 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 01:46:56,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 01:46:56,880 [I] DB_PATCHES have already been applied
I20260811 01:46:56.887056 22361 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-admin:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskw71eKA/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 01:46:56.887159 22361 mini_ranger.cc:193] Using host: 127.21.214.84
I20260811 01:46:56.909787 22361 mini_ranger.cc:227] Ranger admin URL: http://127.21.214.84:60635
Aug 11, 2026 1:46:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 1:46:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 1:46:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 1:46:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 1:46:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 1:46:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 1:46:57 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 01:46:58.188431 22361 mini_ranger.cc:160] Time spent starting Ranger: real 9.548s user 0.003s sys 0.064s
Aug 11, 2026 1:47:20 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 1:47:20 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 1:47:20 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 1:47:20 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 1:47:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 1:47:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 1:47:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 1:47:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 1:47:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 1:47:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 1:47:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 1:47:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 1:47:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 1:47:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 1:47:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 1:47:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 1:47:20 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 1:47:21 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 01:47:21.884174 22361 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.214.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.214.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.214.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/rangerkms_127.21.214.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.214.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/rangerkms_127.21.214.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.214.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.214.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.214.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/HTTP_127.21.214.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.214.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/HTTP_127.21.214.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 01:47:22 dist-test-slave-9gcw krb5kdc[3581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412842, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 01:47:22.117574 22361 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 01:47:22.152797 22361 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 01:47:23.248880 22361 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 01:47:23.249014 22361 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-kms
I20260811 01:47:23.249073 22361 mini_ranger_kms.cc:227] Using postgres at 127.21.214.84:54427
2026-08-11 01:47:23,663 [I] DB FLAVOR :POSTGRES
2026-08-11 01:47:23,664 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 01:47:23,664 [I] Checking connection
2026-08-11 01:47:23,664 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 01:47:23,942 [I] connection success
2026-08-11 01:47:23,942 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 01:47:23,942 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 01:47:23,943 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 01:47:24,208 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 01:47:24,208 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 01:47:24,208 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 01:47:24,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.214.84:54427/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 01:47:24,491 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 01:47:25.039844 22361 mini_ranger_kms.cc:310] Created kms service
I20260811 01:47:25.283327 22361 mini_ranger_kms.cc:326] Created kudu user
I20260811 01:47:25.331836 22361 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 01:47:25 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412842, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
I20260811 01:47:25.769994 22361 mini_ranger_kms.cc:384] Added ranger policy
I20260811 01:47:25.770344 22361 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-kms:/tmp/dist-test-taskw71eKA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskw71eKA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 01:47:25.770404 22361 mini_ranger_kms.cc:240] Using host: 127.21.214.84
I20260811 01:47:25.793335 22361 mini_ranger_kms.cc:276] Ranger KMS PID: 4036
I20260811 01:47:25.793485 22361 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.214.84:34581
Aug 11, 2026 1:47:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 1:47:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 1:47:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 1:47:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 1:47:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 1:47:27 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 01:47:28 dist-test-slave-9gcw krb5kdc[3581](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786412848, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.214.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:28.512226 22361 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.395s user 0.097s sys 0.350s
I20260811 01:47:28.512501 22361 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 01:47:28.512691 22361 mini_ranger_kms.cc:401] 127.21.214.84:34581/kms/v1/keys
Aug 11 01:47:28 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412848, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.214.84@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
Aug 11 01:47:29 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412848, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.214.84@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
Aug 11 01:47:32 dist-test-slave-9gcw krb5kdc[3581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:47:32.747231 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:44727
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:41325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:44727
--ranger_config_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.214.84:34581/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:47:33.210216 4097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:33.210554 4097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:33.210736 4097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:33.219215 4097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:47:33.219672 4097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:33.219970 4097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:33.220105 4097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:47:33.220203 4097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:47:33.233608 4097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:41325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.214.84:34581/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:44727
--ranger_config_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:44727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:33.235270 4097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:33.238094 4097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:33.254760 4105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:33.254783 4102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:33.255198 4103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:33.257444 4097 server_base.cc:1034] running on GCE node
I20260811 01:47:33.258572 4097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:33.260576 4097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:33.261992 4097 hybrid_clock.cc:648] HybridClock initialized: now 1786412853261898 us; error 115 us; skew 500 ppm
I20260811 01:47:33.262422 4097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:33 dist-test-slave-9gcw krb5kdc[3581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412853, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:33.267313 4097 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:47:33.270071 4097 webserver.cc:460] Webserver started at http://127.21.214.126:34069/ using document root <none> and password file <none>
I20260811 01:47:33.271559 4097 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:33.271986 4097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:33.272452 4097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 01:47:33 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412853, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
I20260811 01:47:33.342931 4097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "41109bf7280647e0a028ac25c76801f4"
format_stamp: "Formatted at 2026-08-11 01:47:33 on dist-test-slave-9gcw"
server_key: "cbec40ed7b853f460d1b5afccbe941a3"
server_key_iv: "38bc640521816c322c216f98f8eb32f8"
server_key_version: "kuduclusterkey@0"
I20260811 01:47:33.344198 4097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "41109bf7280647e0a028ac25c76801f4"
format_stamp: "Formatted at 2026-08-11 01:47:33 on dist-test-slave-9gcw"
server_key: "cbec40ed7b853f460d1b5afccbe941a3"
server_key_iv: "38bc640521816c322c216f98f8eb32f8"
server_key_version: "kuduclusterkey@0"
I20260811 01:47:33.352372 4097 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.001s
I20260811 01:47:33.372835 4112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:33.374725 4097 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 01:47:33.375034 4097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "41109bf7280647e0a028ac25c76801f4"
format_stamp: "Formatted at 2026-08-11 01:47:33 on dist-test-slave-9gcw"
server_key: "cbec40ed7b853f460d1b5afccbe941a3"
server_key_iv: "38bc640521816c322c216f98f8eb32f8"
server_key_version: "kuduclusterkey@0"
I20260811 01:47:33.375377 4097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:33.406189 4097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:33.412304 4097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:33.412842 4097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:33.438481 4097 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:44727
I20260811 01:47:33.438491 4170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:44727 every 8 connection(s)
I20260811 01:47:33.440091 4097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:47:33.444573 4171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:47:33.449679 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 4097
I20260811 01:47:33.449980 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:47:33.451386 22361 external_mini_cluster.cc:1428] Setting key e1c66ac751af156c273170d6e1c36b89
I20260811 01:47:33.463367 4171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4: Bootstrap starting.
I20260811 01:47:33.469976 4171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4: Neither blocks nor log segments found. Creating new log.
I20260811 01:47:33.471588 4171 log.cc:824] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4: Log is configured to *not* fsync() on all Append() calls
I20260811 01:47:33.475366 4171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4: No bootstrap required, opened a new log
Aug 11 01:47:33 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:47:33.485360 4171 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41109bf7280647e0a028ac25c76801f4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44727 } }
I20260811 01:47:33.486063 4171 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:47:33.486274 4171 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41109bf7280647e0a028ac25c76801f4, State: Initialized, Role: FOLLOWER
I20260811 01:47:33.487107 4171 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41109bf7280647e0a028ac25c76801f4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44727 } }
I20260811 01:47:33.487401 4171 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:47:33.487586 4171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:47:33.487811 4171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:47:33.491890 4171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41109bf7280647e0a028ac25c76801f4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44727 } }
I20260811 01:47:33.492784 4171 leader_election.cc:302] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41109bf7280647e0a028ac25c76801f4; no voters:
I20260811 01:47:33.493769 4171 leader_election.cc:288] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:47:33.493997 4177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:47:33.494974 4177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [term 1 LEADER]: Becoming Leader. State: Replica: 41109bf7280647e0a028ac25c76801f4, State: Running, Role: LEADER
I20260811 01:47:33.495765 4177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41109bf7280647e0a028ac25c76801f4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44727 } }
I20260811 01:47:33.496768 4175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:33.454415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57478 (local address 127.21.214.126:44727)
0811 01:47:33.468312 (+ 13897us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:33.468345 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:33.468430 (+ 85us) server_negotiation.cc:407] Connection header received
0811 01:47:33.471299 (+ 2869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:33.471338 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:33.471920 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:33.472512 (+ 592us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:33.474245 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:33.475988 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:33.477696 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:33.478564 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:33.478892 (+ 328us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:33.484514 (+ 5622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:33.484525 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:33.484546 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:33.484598 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:33.488797 (+ 4199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:33.490117 (+ 1320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:33.490131 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:33.490140 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:33.490681 (+ 541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:33.491256 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:33.491262 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:33.491265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:33.492284 (+ 1019us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:33.492710 (+ 426us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:33.493293 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 01:47:33.493997 (+ 704us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13560,"thread_start_us":13223,"threads_started":1}
I20260811 01:47:33.499115 4180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41109bf7280647e0a028ac25c76801f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41109bf7280647e0a028ac25c76801f4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44727 } } }
I20260811 01:47:33.499576 4180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [sys.catalog]: This master's current role is: LEADER
I20260811 01:47:33.499840 4171 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:47:33.503830 4182 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:47:33.503614 4179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41109bf7280647e0a028ac25c76801f4. Latest consensus state: current_term: 1 leader_uuid: "41109bf7280647e0a028ac25c76801f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41109bf7280647e0a028ac25c76801f4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44727 } } }
I20260811 01:47:33.504122 4179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [sys.catalog]: This master's current role is: LEADER
I20260811 01:47:33.511929 4182 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:47:33.523679 4171 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 01:47:33.527120 4182 catalog_manager.cc:1347] Generated new cluster ID: 5710603c1a4a4ae5bbb0b4625250459b
I20260811 01:47:33.527287 4182 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:47:33.549654 4182 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:47:33.551815 4182 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:47:33.576326 4182 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4: Generated new TSK 0
I20260811 01:47:33.577608 4182 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
I20260811 01:47:34.444248 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.214.84:34581/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:44727
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:41325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 01:47:34 dist-test-slave-9gcw krb5kdc[3581](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786412854, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 01:47:34.972250 4223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:34.972765 4223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:34.972965 4223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:34.985913 4223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:34.986204 4223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:34.986517 4223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:47:35.007398 4223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:41325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.214.84:34581/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:44727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:35.009946 4223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:35.012185 4223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:35.033567 4238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:35.034636 4236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:35.035461 4235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:35.036464 4223 server_base.cc:1034] running on GCE node
I20260811 01:47:35.037384 4223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:35.039175 4223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:35.042624 4223 hybrid_clock.cc:648] HybridClock initialized: now 1786412855042446 us; error 171 us; skew 500 ppm
I20260811 01:47:35.043190 4223 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:35 dist-test-slave-9gcw krb5kdc[3581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412855, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:35.051927 4223 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:47:35.058561 4223 webserver.cc:460] Webserver started at http://127.21.214.65:41165/ using document root <none> and password file <none>
I20260811 01:47:35.059900 4223 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:35.060083 4223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:35.060520 4223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 01:47:35 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412855, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
I20260811 01:47:35.097783 4223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "3676f980fd014d92a8d93ac0764bb4d4"
format_stamp: "Formatted at 2026-08-11 01:47:35 on dist-test-slave-9gcw"
server_key: "f411dcd5eeecff25be13a689653a7ff2"
server_key_iv: "654cb879214d61ec52cd08598b828db3"
server_key_version: "kuduclusterkey@0"
I20260811 01:47:35.100024 4223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "3676f980fd014d92a8d93ac0764bb4d4"
format_stamp: "Formatted at 2026-08-11 01:47:35 on dist-test-slave-9gcw"
server_key: "f411dcd5eeecff25be13a689653a7ff2"
server_key_iv: "654cb879214d61ec52cd08598b828db3"
server_key_version: "kuduclusterkey@0"
I20260811 01:47:35.110915 4223 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.011s sys 0.001s
I20260811 01:47:35.144490 4247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:35.147784 4223 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.002s sys 0.001s
I20260811 01:47:35.148150 4223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "3676f980fd014d92a8d93ac0764bb4d4"
format_stamp: "Formatted at 2026-08-11 01:47:35 on dist-test-slave-9gcw"
server_key: "f411dcd5eeecff25be13a689653a7ff2"
server_key_iv: "654cb879214d61ec52cd08598b828db3"
server_key_version: "kuduclusterkey@0"
I20260811 01:47:35.148531 4223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:35.193948 4223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:35.201903 4223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:35.202322 4223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:35.204114 4223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:47:35.206240 4223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:47:35.206358 4223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:35.206545 4223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:47:35.206667 4223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:35.265198 4372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:46155 every 8 connection(s)
I20260811 01:47:35.265192 4223 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:46155
I20260811 01:47:35.267868 4223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:47:35.275445 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 4223
I20260811 01:47:35.275694 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:47:35.276568 22361 external_mini_cluster.cc:1428] Setting key de3bf6ffc4c6d50f94398ca34f1055d8
Aug 11 01:47:35 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412855, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.214.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.66@KRBTEST.COM" created.
I20260811 01:47:35.338897 4376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:35.285911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:37645 (local address 127.21.214.126:44727)
0811 01:47:35.306170 (+ 20259us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:35.306182 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:35.306611 (+ 429us) server_negotiation.cc:407] Connection header received
0811 01:47:35.309898 (+ 3287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:35.309905 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:35.310051 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:35.310150 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:35.317546 (+ 7396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:35.318793 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:35.322024 (+ 3231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:35.322764 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:35.322888 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:35.329169 (+ 6281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:35.329192 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:35.329198 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:35.329249 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:35.333711 (+ 4462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:35.334976 (+ 1265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:35.334983 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:35.334987 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:35.335298 (+ 311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:35.335982 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:35.335988 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:35.335991 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:35.336415 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:35.336567 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:35.338166 (+ 1599us) server_negotiation.cc:299] Negotiation successful
0811 01:47:35.338492 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20024,"thread_start_us":20974,"threads_started":1}
I20260811 01:47:35.341512 4375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:35.284970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:44727 (local address 127.21.214.65:37645)
0811 01:47:35.306317 (+ 21347us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:35.306399 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:35.309330 (+ 2931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:35.311270 (+ 1940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:35.311294 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:35.312384 (+ 1090us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:35.317209 (+ 4825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:35.317256 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:35.319078 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:35.319084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:35.321652 (+ 2568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:35.321740 (+ 88us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:35.323370 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:35.323382 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:35.323776 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:35.325419 (+ 1643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:35.325457 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:35.328779 (+ 3322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:35.333981 (+ 5202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:35.333994 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:35.334017 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:35.334790 (+ 773us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:35.335510 (+ 720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:35.335516 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:35.335519 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:35.335825 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:35.336677 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:35.336686 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:35.337540 (+ 854us) client_negotiation.cc:769] Sending connection context
0811 01:47:35.339153 (+ 1613us) client_negotiation.cc:240] Negotiation successful
0811 01:47:35.339805 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20873,"thread_start_us":20471,"threads_started":1}
I20260811 01:47:35.344127 4373 heartbeater.cc:347] Connected to a master server at 127.21.214.126:44727
I20260811 01:47:35.344611 4373 heartbeater.cc:464] Registering TS with master...
I20260811 01:47:35.345855 4373 heartbeater.cc:511] Master 127.21.214.126:44727 requested a full tablet report, sending...
Aug 11 01:47:35 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412854, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
I20260811 01:47:35.349246 4130 ts_manager.cc:194] Registered new tserver with Master: 3676f980fd014d92a8d93ac0764bb4d4 (127.21.214.65:46155)
I20260811 01:47:35.351468 4130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:37645
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
W20260811 01:47:35.403215 22361 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.21.214.66: real 0.123s user 0.000s sys 0.025s
I20260811 01:47:35.404320 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.214.84:34581/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:44727
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:41325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 01:47:35 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412854, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
W20260811 01:47:35.946710 4382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:35.947304 4382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:35.947568 4382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:35.960522 4382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:35.960701 4382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:35.960901 4382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:47:35.979104 4382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:41325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.214.84:34581/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:44727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:35.981458 4382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:35.983526 4382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:36.028049 4390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:36.031416 4389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:36.032900 4392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:36.039923 4382 server_base.cc:1034] running on GCE node
I20260811 01:47:36.041033 4382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:36.042608 4382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:36.044118 4382 hybrid_clock.cc:648] HybridClock initialized: now 1786412856044028 us; error 84 us; skew 500 ppm
I20260811 01:47:36.044644 4382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:36 dist-test-slave-9gcw krb5kdc[3581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412856, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:36.050063 4382 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:47:36.052942 4382 webserver.cc:460] Webserver started at http://127.21.214.66:33751/ using document root <none> and password file <none>
I20260811 01:47:36.054454 4382 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:36.054718 4382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:36.055251 4382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 01:47:36 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412856, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
I20260811 01:47:36.085335 4382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "b485ed768d0a40d7bf2468232a394a82"
format_stamp: "Formatted at 2026-08-11 01:47:36 on dist-test-slave-9gcw"
server_key: "8deccc6100d51cd2b09b64ca3d5df2b9"
server_key_iv: "dabd70b614acf84b8a0d58eac5bf5448"
server_key_version: "kuduclusterkey@0"
I20260811 01:47:36.087055 4382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "b485ed768d0a40d7bf2468232a394a82"
format_stamp: "Formatted at 2026-08-11 01:47:36 on dist-test-slave-9gcw"
server_key: "8deccc6100d51cd2b09b64ca3d5df2b9"
server_key_iv: "dabd70b614acf84b8a0d58eac5bf5448"
server_key_version: "kuduclusterkey@0"
I20260811 01:47:36.096961 4382 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.004s
I20260811 01:47:36.118824 4408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:36.121222 4382 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 01:47:36.121502 4382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "b485ed768d0a40d7bf2468232a394a82"
format_stamp: "Formatted at 2026-08-11 01:47:36 on dist-test-slave-9gcw"
server_key: "8deccc6100d51cd2b09b64ca3d5df2b9"
server_key_iv: "dabd70b614acf84b8a0d58eac5bf5448"
server_key_version: "kuduclusterkey@0"
I20260811 01:47:36.121812 4382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:36.161525 4382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:36.167433 4382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:36.167817 4382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:36.169309 4382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:47:36.171622 4382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:47:36.171725 4382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:36.171867 4382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:47:36.172001 4382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:36.222091 4382 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:42629
I20260811 01:47:36.222095 4536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:42629 every 8 connection(s)
I20260811 01:47:36.224287 4382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:47:36.229301 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 4382
I20260811 01:47:36.229542 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:47:36.230417 22361 external_mini_cluster.cc:1428] Setting key a7c6e64b2aff36f89ab14ee01777d893
Aug 11 01:47:36 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412856, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:47:36.250188 4539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:36.228362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:46521 (local address 127.21.214.126:44727)
0811 01:47:36.231591 (+ 3229us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:36.231603 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:36.231629 (+ 26us) server_negotiation.cc:407] Connection header received
0811 01:47:36.233557 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:36.233563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:36.233650 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:36.233789 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:36.235381 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:36.236224 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:36.237974 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:36.238387 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:36.238530 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:36.242373 (+ 3843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:36.242378 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:36.242381 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:36.242418 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:36.245316 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:36.246307 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:36.246313 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:36.246317 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:36.246493 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:36.247212 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:36.247218 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:36.247222 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:36.247756 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:36.247983 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:36.249385 (+ 1402us) server_negotiation.cc:299] Negotiation successful
0811 01:47:36.249653 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3024,"thread_start_us":382,"threads_started":1}
I20260811 01:47:36.251462 4540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:36.229200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:44727 (local address 127.21.214.66:46521)
0811 01:47:36.230368 (+ 1168us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:36.230439 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:36.233266 (+ 2827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:36.234063 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:36.234073 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:36.234555 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:36.235190 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:36.235209 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:36.236394 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:36.236399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:36.237812 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:36.237824 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:36.238630 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:36.238639 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:36.238889 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:36.239642 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:36.239669 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:36.242114 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:36.245467 (+ 3353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:36.245477 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:36.245497 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:36.246118 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:36.246628 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:36.246634 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:36.246640 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:36.247005 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:36.248070 (+ 1065us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:36.248080 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:36.248550 (+ 470us) client_negotiation.cc:769] Sending connection context
0811 01:47:36.249017 (+ 467us) client_negotiation.cc:240] Negotiation successful
0811 01:47:36.249686 (+ 669us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"thread_start_us":486,"threads_started":1}
I20260811 01:47:36.253374 4537 heartbeater.cc:347] Connected to a master server at 127.21.214.126:44727
I20260811 01:47:36.253756 4537 heartbeater.cc:464] Registering TS with master...
I20260811 01:47:36.254855 4537 heartbeater.cc:511] Master 127.21.214.126:44727 requested a full tablet report, sending...
I20260811 01:47:36.256771 4130 ts_manager.cc:194] Registered new tserver with Master: b485ed768d0a40d7bf2468232a394a82 (127.21.214.66:42629)
I20260811 01:47:36.257797 4130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:46521
WARNING: no policy specified for kudu/127.21.214.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
I20260811 01:47:36.310369 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.214.84:34581/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:44727
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:41325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:47:36.356591 4373 heartbeater.cc:503] Master 127.21.214.126:44727 was elected leader, sending a full tablet report...
W20260811 01:47:36.721539 4545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:36.721932 4545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:36.722234 4545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:36.731536 4545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:36.731808 4545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:36.732115 4545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:47:36.745414 4545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:41325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.214.84:34581/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:44727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:36.747151 4545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:36.748840 4545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:36.766656 4552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:36.766495 4551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:36.767364 4555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:36.768529 4545 server_base.cc:1034] running on GCE node
I20260811 01:47:36.769371 4545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:36.770640 4545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:36.771914 4545 hybrid_clock.cc:648] HybridClock initialized: now 1786412856771851 us; error 57 us; skew 500 ppm
I20260811 01:47:36.772332 4545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:36 dist-test-slave-9gcw krb5kdc[3581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412856, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:36.776696 4545 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:47:36.778970 4545 webserver.cc:460] Webserver started at http://127.21.214.67:41927/ using document root <none> and password file <none>
I20260811 01:47:36.779884 4545 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:36.780062 4545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:36.780463 4545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 01:47:36 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412856, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
I20260811 01:47:36.800918 4545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "b6ec958527af488f930237622caa9a80"
format_stamp: "Formatted at 2026-08-11 01:47:36 on dist-test-slave-9gcw"
server_key: "3cd83f0d39356e1a5cb932ff3b06dbbd"
server_key_iv: "4b608547a65a8251d0982b6968bc3203"
server_key_version: "kuduclusterkey@0"
I20260811 01:47:36.802243 4545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "b6ec958527af488f930237622caa9a80"
format_stamp: "Formatted at 2026-08-11 01:47:36 on dist-test-slave-9gcw"
server_key: "3cd83f0d39356e1a5cb932ff3b06dbbd"
server_key_iv: "4b608547a65a8251d0982b6968bc3203"
server_key_version: "kuduclusterkey@0"
I20260811 01:47:36.811878 4545 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.009s sys 0.000s
I20260811 01:47:36.833350 4562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:36.835974 4545 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 01:47:36.836211 4545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "b6ec958527af488f930237622caa9a80"
format_stamp: "Formatted at 2026-08-11 01:47:36 on dist-test-slave-9gcw"
server_key: "3cd83f0d39356e1a5cb932ff3b06dbbd"
server_key_iv: "4b608547a65a8251d0982b6968bc3203"
server_key_version: "kuduclusterkey@0"
I20260811 01:47:36.836556 4545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:36.893373 4545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:36.898599 4545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:36.899001 4545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:36.900339 4545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:47:36.902217 4545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:47:36.902302 4545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:36.902489 4545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:47:36.902547 4545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:36.961402 4545 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:35229
I20260811 01:47:36.961442 4689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:35229 every 8 connection(s)
I20260811 01:47:36.962955 4545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:47:36.971246 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 4545
I20260811 01:47:36.971567 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:47:36.972148 22361 external_mini_cluster.cc:1428] Setting key 16f21527131f4430769318d5112cf197
Aug 11 01:47:36 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412856, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:47:36.990213 4692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:36.966376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:52777 (local address 127.21.214.126:44727)
0811 01:47:36.966941 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:36.966947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:36.969247 (+ 2300us) server_negotiation.cc:407] Connection header received
0811 01:47:36.971097 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:36.971106 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:36.971236 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:36.971370 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:36.973350 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:36.974958 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:36.977412 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:36.977900 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:36.978056 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:36.982537 (+ 4481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:36.982542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:36.982546 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:36.982581 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:36.985525 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:36.986592 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:36.986596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:36.986599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:36.986763 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:36.987539 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:36.987544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:36.987549 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:36.987975 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:36.988129 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:36.989184 (+ 1055us) server_negotiation.cc:299] Negotiation successful
0811 01:47:36.989720 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":318,"threads_started":1}
I20260811 01:47:36.991201 4693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:36.966969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:44727 (local address 127.21.214.67:52777)
0811 01:47:36.968522 (+ 1553us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:36.968616 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:36.970671 (+ 2055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:36.971668 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:36.971678 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:36.972182 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:36.973062 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:36.973109 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:36.975157 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:36.975162 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:36.977228 (+ 2066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:36.977273 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:36.978170 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:36.978181 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:36.978467 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:36.979246 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:36.979281 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:36.982235 (+ 2954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:36.985689 (+ 3454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:36.985700 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:36.985722 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:36.986414 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:36.986897 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:36.986903 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:36.986907 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:36.987226 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:36.988197 (+ 971us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:36.988205 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:36.988673 (+ 468us) client_negotiation.cc:769] Sending connection context
0811 01:47:36.989070 (+ 397us) client_negotiation.cc:240] Negotiation successful
0811 01:47:36.989831 (+ 761us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":943,"thread_start_us":469,"threads_started":1}
I20260811 01:47:36.992789 4690 heartbeater.cc:347] Connected to a master server at 127.21.214.126:44727
I20260811 01:47:36.993217 4690 heartbeater.cc:464] Registering TS with master...
I20260811 01:47:36.994212 4690 heartbeater.cc:511] Master 127.21.214.126:44727 requested a full tablet report, sending...
I20260811 01:47:36.996042 4130 ts_manager.cc:194] Registered new tserver with Master: b6ec958527af488f930237622caa9a80 (127.21.214.67:35229)
I20260811 01:47:36.997311 4130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:52777
I20260811 01:47:37.002580 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:47:37.004632 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 4223
Aug 11 01:47:37 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:47:37.028091 4696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:37.005877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44492 (local address 127.21.214.65:46155)
0811 01:47:37.009860 (+ 3983us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:37.009871 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:37.010335 (+ 464us) server_negotiation.cc:407] Connection header received
0811 01:47:37.010568 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:37.010579 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:37.010827 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:37.011017 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:37.012796 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:37.014110 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:37.015744 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:37.016361 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:37.016534 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:37.021325 (+ 4791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:37.021337 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:37.021353 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:37.021389 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:37.024817 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:37.025717 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:37.025724 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:37.025726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:37.025871 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:37.026294 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:37.026298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:37.026300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:37.026654 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:37.027055 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:37.027389 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 01:47:37.027728 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3742,"thread_start_us":3546,"threads_started":1}
I20260811 01:47:37.030282 4367 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:37.121230 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 4223
I20260811 01:47:37.155262 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 4382
Aug 11 01:47:37 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.66@KRBTEST.COM
I20260811 01:47:37.182009 4699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:37.163654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39868 (local address 127.21.214.66:42629)
0811 01:47:37.164247 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:37.164256 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:37.164296 (+ 40us) server_negotiation.cc:407] Connection header received
0811 01:47:37.164381 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:37.164388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:37.164660 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:37.164883 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:37.166260 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:37.167367 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:37.168930 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:37.169799 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:37.169958 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:37.175196 (+ 5238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:37.175210 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:37.175227 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:37.175266 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:37.178571 (+ 3305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:37.179451 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:37.179461 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:37.179464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:37.179623 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:37.180088 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:37.180094 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:37.180097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:37.180572 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:37.180924 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:37.181278 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 01:47:37.181628 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":279,"threads_started":1}
I20260811 01:47:37.183574 4530 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:37.271404 4537 heartbeater.cc:503] Master 127.21.214.126:44727 was elected leader, sending a full tablet report...
I20260811 01:47:37.273399 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 4382
W20260811 01:47:37.303197 4115 connection.cc:535] server connection from 127.21.214.66:46521 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 01:47:37.303479 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 4545
Aug 11 01:47:37 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.67@KRBTEST.COM
I20260811 01:47:37.323357 4703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:37.304739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33830 (local address 127.21.214.67:35229)
0811 01:47:37.305585 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:37.305593 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:37.305651 (+ 58us) server_negotiation.cc:407] Connection header received
0811 01:47:37.305746 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:37.305758 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:37.306002 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:37.306144 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:37.307634 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:37.308911 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:37.310413 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:37.311128 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:37.311303 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:37.316388 (+ 5085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:37.316402 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:37.316419 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:37.316462 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:37.320127 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:37.320949 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:37.320957 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:37.320960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:37.321139 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:37.321622 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:37.321626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:37.321628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:37.322091 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:37.322362 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:37.322662 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 01:47:37.323035 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":508,"threads_started":1}
I20260811 01:47:37.324846 4685 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:37.419173 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 4545
I20260811 01:47:37.440054 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 4097
I20260811 01:47:37.441495 4166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:37.556470 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 4097
I20260811 01:47:37.586104 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:44727
--webserver_interface=127.21.214.126
--webserver_port=34069
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:41325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:44727
--ranger_config_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.214.84:34581/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.214.84:34581/kms with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:47:37.939698 4706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:37.940120 4706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:37.940303 4706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:37.948017 4706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:47:37.948192 4706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:37.948328 4706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:37.948431 4706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:47:37.948521 4706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:47:37.960750 4706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:41325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.214.84:34581/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:44727
--ranger_config_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:44727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=34069
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:37.962404 4706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:37.964145 4706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:37.978988 4712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:37.978997 4717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:37.979054 4714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:37.980476 4706 server_base.cc:1034] running on GCE node
I20260811 01:47:37.981604 4706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:37.983418 4706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:37.984684 4706 hybrid_clock.cc:648] HybridClock initialized: now 1786412857984604 us; error 98 us; skew 500 ppm
I20260811 01:47:37.985203 4706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:37 dist-test-slave-9gcw krb5kdc[3581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412857, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:37.992846 4706 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:47:37.994709 4706 webserver.cc:460] Webserver started at http://127.21.214.126:34069/ using document root <none> and password file <none>
I20260811 01:47:37.995522 4706 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:37.995702 4706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 01:47:38 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412857, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
I20260811 01:47:38.075304 4706 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 01:47:38.078975 4725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:38.080646 4706 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 01:47:38.080827 4706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "41109bf7280647e0a028ac25c76801f4"
format_stamp: "Formatted at 2026-08-11 01:47:33 on dist-test-slave-9gcw"
server_key: "cbec40ed7b853f460d1b5afccbe941a3"
server_key_iv: "38bc640521816c322c216f98f8eb32f8"
server_key_version: "kuduclusterkey@0"
I20260811 01:47:38.082037 4706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:38.108175 4706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:38.112423 4706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:38.112696 4706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:38.132704 4706 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:44727
I20260811 01:47:38.132711 4782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:44727 every 8 connection(s)
I20260811 01:47:38.134359 4706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:47:38.138469 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 4706
I20260811 01:47:38.139605 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.214.84:34581/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:46155
--local_ip_for_outbound_sockets=127.21.214.65
--tserver_master_addrs=127.21.214.126:44727
--webserver_port=41165
--webserver_interface=127.21.214.65
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:41325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.214.84:34581/kms with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:47:38.143687 4783 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:47:38.149777 4783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4: Bootstrap starting.
I20260811 01:47:38.170854 4783 log.cc:824] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4: Log is configured to *not* fsync() on all Append() calls
I20260811 01:47:38.181049 4783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:47:38.181806 4783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4: Bootstrap complete.
I20260811 01:47:38.191527 4783 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41109bf7280647e0a028ac25c76801f4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44727 } }
I20260811 01:47:38.192624 4783 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41109bf7280647e0a028ac25c76801f4, State: Initialized, Role: FOLLOWER
I20260811 01:47:38.193392 4783 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41109bf7280647e0a028ac25c76801f4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44727 } }
I20260811 01:47:38.193630 4783 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:47:38.193770 4783 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:47:38.193970 4783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:47:38.198133 4783 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41109bf7280647e0a028ac25c76801f4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44727 } }
I20260811 01:47:38.198655 4783 leader_election.cc:302] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41109bf7280647e0a028ac25c76801f4; no voters:
I20260811 01:47:38.199335 4783 leader_election.cc:288] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 01:47:38.199489 4788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 01:47:38.200886 4788 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [term 2 LEADER]: Becoming Leader. State: Replica: 41109bf7280647e0a028ac25c76801f4, State: Running, Role: LEADER
I20260811 01:47:38.201567 4788 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41109bf7280647e0a028ac25c76801f4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44727 } }
I20260811 01:47:38.201792 4783 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:47:38.204484 4789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "41109bf7280647e0a028ac25c76801f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41109bf7280647e0a028ac25c76801f4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44727 } } }
I20260811 01:47:38.205039 4789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [sys.catalog]: This master's current role is: LEADER
I20260811 01:47:38.209021 4790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41109bf7280647e0a028ac25c76801f4. Latest consensus state: current_term: 2 leader_uuid: "41109bf7280647e0a028ac25c76801f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41109bf7280647e0a028ac25c76801f4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44727 } } }
I20260811 01:47:38.209551 4790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4 [sys.catalog]: This master's current role is: LEADER
I20260811 01:47:38.211048 4795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:47:38.219125 4795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:47:38.220029 4783 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 01:47:38.220903 4795 catalog_manager.cc:1259] Loaded cluster ID: 5710603c1a4a4ae5bbb0b4625250459b
I20260811 01:47:38.221074 4795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:47:38.226320 4795 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:47:38.228420 4795 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4: Loaded TSK: 0
I20260811 01:47:38.230649 4795 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 01:47:38.636338 4786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:38.636888 4786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:38.637323 4786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:38.649704 4786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:38.649964 4786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:38.650307 4786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:47:38.667089 4786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:41325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.214.84:34581/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:46155
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=41165
--tserver_master_addrs=127.21.214.126:44727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:38.669214 4786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:38.671247 4786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:38.693595 4829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:38.701700 4830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:38.703447 4833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:38.704130 4786 server_base.cc:1034] running on GCE node
I20260811 01:47:38.705103 4786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:38.718390 4786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:38.719828 4786 hybrid_clock.cc:648] HybridClock initialized: now 1786412858719728 us; error 102 us; skew 500 ppm
I20260811 01:47:38.720417 4786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:38 dist-test-slave-9gcw krb5kdc[3581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412858, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:38.731069 4786 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:47:38.758272 4786 webserver.cc:460] Webserver started at http://127.21.214.65:41165/ using document root <none> and password file <none>
I20260811 01:47:38.759527 4786 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:38.759747 4786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 01:47:38 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412858, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
I20260811 01:47:38.815119 4786 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 01:47:38.831132 4841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:38.833616 4786 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.009s sys 0.008s
I20260811 01:47:38.833942 4786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "3676f980fd014d92a8d93ac0764bb4d4"
format_stamp: "Formatted at 2026-08-11 01:47:35 on dist-test-slave-9gcw"
server_key: "f411dcd5eeecff25be13a689653a7ff2"
server_key_iv: "654cb879214d61ec52cd08598b828db3"
server_key_version: "kuduclusterkey@0"
I20260811 01:47:38.835513 4786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:38.864338 4786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:38.870184 4786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:38.870533 4786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:38.871884 4786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:47:38.873721 4786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:47:38.873828 4786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:38.873996 4786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:47:38.874092 4786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:38.952913 4786 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:46155
I20260811 01:47:38.952967 4977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:46155 every 8 connection(s)
I20260811 01:47:38.954638 4786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:47:38.964670 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 4786
I20260811 01:47:38.965690 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.214.84:34581/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:42629
--local_ip_for_outbound_sockets=127.21.214.66
--tserver_master_addrs=127.21.214.126:44727
--webserver_port=33751
--webserver_interface=127.21.214.66
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:41325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.214.84:34581/kms with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 01:47:38 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412858, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:47:39.000697 4981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:38.958401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:44727 (local address 127.21.214.65:55795)
0811 01:47:38.959392 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:38.959468 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:38.962856 (+ 3388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:38.965853 (+ 2997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:38.965869 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:38.966628 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:38.967567 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:38.967600 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:38.975544 (+ 7944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:38.975555 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:38.978303 (+ 2748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:38.978336 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:38.980643 (+ 2307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:38.980657 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:38.981039 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:38.982220 (+ 1181us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:38.982263 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:38.985979 (+ 3716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:38.992098 (+ 6119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:38.992115 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:38.992136 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:38.992948 (+ 812us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:38.995207 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:38.995216 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:38.995221 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:38.995646 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:38.997142 (+ 1496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:38.997157 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:38.997740 (+ 583us) client_negotiation.cc:769] Sending connection context
0811 01:47:38.998184 (+ 444us) client_negotiation.cc:240] Negotiation successful
0811 01:47:38.998915 (+ 731us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":370,"threads_started":1}
I20260811 01:47:39.000895 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:38.958315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:55795 (local address 127.21.214.126:44727)
0811 01:47:38.961113 (+ 2798us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:38.961139 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:38.961228 (+ 89us) server_negotiation.cc:407] Connection header received
0811 01:47:38.963592 (+ 2364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:38.963619 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:38.964343 (+ 724us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:38.964937 (+ 594us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:38.967920 (+ 2983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:38.972775 (+ 4855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:38.978698 (+ 5923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:38.980184 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:38.980502 (+ 318us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:38.986489 (+ 5987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:38.986506 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:38.986535 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:38.986608 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:38.991798 (+ 5190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:38.994509 (+ 2711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:38.994524 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:38.994539 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:38.994958 (+ 419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:38.995853 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:38.995863 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:38.995870 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:38.996866 (+ 996us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:38.997234 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:38.998639 (+ 1405us) server_negotiation.cc:299] Negotiation successful
0811 01:47:38.999205 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2467,"thread_start_us":2197,"threads_started":1}
I20260811 01:47:39.005782 4978 heartbeater.cc:347] Connected to a master server at 127.21.214.126:44727
I20260811 01:47:39.006259 4978 heartbeater.cc:464] Registering TS with master...
I20260811 01:47:39.007468 4978 heartbeater.cc:511] Master 127.21.214.126:44727 requested a full tablet report, sending...
W20260811 01:47:39.011924 4978 heartbeater.cc:644] Failed to heartbeat to 127.21.214.126:44727 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 01:47:39.012494 4978 heartbeater.cc:464] Registering TS with master...
I20260811 01:47:39.013036 4978 heartbeater.cc:511] Master 127.21.214.126:44727 requested a full tablet report, sending...
I20260811 01:47:39.014611 4978 heartbeater.cc:464] Registering TS with master...
I20260811 01:47:39.014915 4978 heartbeater.cc:511] Master 127.21.214.126:44727 requested a full tablet report, sending...
W20260811 01:47:39.016541 4978 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 01:47:39.415989 4982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:39.416350 4982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:39.416525 4982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:39.426218 4982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:39.426389 4982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:39.426636 4982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:47:39.443035 4982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:41325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.214.84:34581/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:42629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=33751
--tserver_master_addrs=127.21.214.126:44727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:39.445194 4982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:39.447418 4982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:39.479852 4999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:39.480265 5001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:39.483309 4998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:39.485600 4982 server_base.cc:1034] running on GCE node
I20260811 01:47:39.486711 4982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:39.497524 4982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:39.501066 4982 hybrid_clock.cc:648] HybridClock initialized: now 1786412859500969 us; error 97 us; skew 500 ppm
I20260811 01:47:39.501582 4982 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:39 dist-test-slave-9gcw krb5kdc[3581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412859, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:39.506803 4982 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:47:39.509200 4982 webserver.cc:460] Webserver started at http://127.21.214.66:33751/ using document root <none> and password file <none>
I20260811 01:47:39.510263 4982 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:39.510437 4982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 01:47:39 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412859, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
I20260811 01:47:39.542016 4982 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 01:47:39.546242 5010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:39.548269 4982 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 01:47:39.548537 4982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "b485ed768d0a40d7bf2468232a394a82"
format_stamp: "Formatted at 2026-08-11 01:47:36 on dist-test-slave-9gcw"
server_key: "8deccc6100d51cd2b09b64ca3d5df2b9"
server_key_iv: "dabd70b614acf84b8a0d58eac5bf5448"
server_key_version: "kuduclusterkey@0"
I20260811 01:47:39.550221 4982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 11 01:47:39 dist-test-slave-9gcw krb5kdc[3581](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786412859, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:39.567945 4982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:39.573868 4982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:39.574359 4982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:39.576671 4982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:47:39.578712 4982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:47:39.578891 4982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 01:47:39.579062 4982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:47:39.579192 4982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:39.644990 4982 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:42629
I20260811 01:47:39.645043 5136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:42629 every 8 connection(s)
I20260811 01:47:39.647444 4982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:47:39.648988 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 4982
I20260811 01:47:39.649832 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.214.84:34581/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:35229
--local_ip_for_outbound_sockets=127.21.214.67
--tserver_master_addrs=127.21.214.126:44727
--webserver_port=41927
--webserver_interface=127.21.214.67
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:41325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.214.84:34581/kms with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 01:47:39 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412859, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:47:39.682207 5139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:39.650692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:60797 (local address 127.21.214.126:44727)
0811 01:47:39.651292 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:39.651299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:39.652497 (+ 1198us) server_negotiation.cc:407] Connection header received
0811 01:47:39.655315 (+ 2818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:39.655325 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:39.655445 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:39.655584 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:39.661014 (+ 5430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:39.662071 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:39.664898 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:39.665742 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:39.665888 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:39.670663 (+ 4775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:39.670673 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:39.670681 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:39.670747 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:39.676817 (+ 6070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:39.678133 (+ 1316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:39.678140 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:39.678144 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:39.678304 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:39.678926 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:39.678931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:39.678934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:39.679344 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:39.679475 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:39.681521 (+ 2046us) server_negotiation.cc:299] Negotiation successful
0811 01:47:39.681793 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":276,"threads_started":1}
I20260811 01:47:39.685554 5140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:39.651414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:44727 (local address 127.21.214.66:60797)
0811 01:47:39.652297 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:39.652347 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:39.654905 (+ 2558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:39.655892 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:39.655906 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:39.656561 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:39.660713 (+ 4152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:39.660749 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:39.662298 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:39.662304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:39.664675 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:39.664692 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:39.666034 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:39.666042 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:39.666383 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:39.667290 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:39.667331 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:39.670332 (+ 3001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:39.677048 (+ 6716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:39.677062 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:39.677084 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:39.677939 (+ 855us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:39.678469 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:39.678474 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:39.678478 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:39.678778 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:39.679583 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:39.679591 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:39.680083 (+ 492us) client_negotiation.cc:769] Sending connection context
0811 01:47:39.682852 (+ 2769us) client_negotiation.cc:240] Negotiation successful
0811 01:47:39.683568 (+ 716us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":389,"threads_started":1}
I20260811 01:47:39.687870 5137 heartbeater.cc:347] Connected to a master server at 127.21.214.126:44727
I20260811 01:47:39.688251 5137 heartbeater.cc:464] Registering TS with master...
I20260811 01:47:39.689201 5137 heartbeater.cc:511] Master 127.21.214.126:44727 requested a full tablet report, sending...
I20260811 01:47:39.691607 4745 ts_manager.cc:194] Registered new tserver with Master: b485ed768d0a40d7bf2468232a394a82 (127.21.214.66:42629)
I20260811 01:47:39.694065 4745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:60797
I20260811 01:47:40.016788 4978 heartbeater.cc:464] Registering TS with master...
I20260811 01:47:40.017868 4978 heartbeater.cc:511] Master 127.21.214.126:44727 requested a full tablet report, sending...
I20260811 01:47:40.020035 4745 ts_manager.cc:194] Registered new tserver with Master: 3676f980fd014d92a8d93ac0764bb4d4 (127.21.214.65:46155)
I20260811 01:47:40.022208 4745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:55795
Aug 11 01:47:40 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412859, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
W20260811 01:47:40.195758 5141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:40.196280 5141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:40.196491 5141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:40.212693 5141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:40.213188 5141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:40.213724 5141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:47:40.235458 5141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:41325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.214.84:34581/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:35229
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=41927
--tserver_master_addrs=127.21.214.126:44727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:40.238113 5141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:40.240581 5141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:40.266202 5147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:40.266273 5148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:40.272617 5151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:40.274923 5141 server_base.cc:1034] running on GCE node
I20260811 01:47:40.276054 5141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:40.277737 5141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:40.279295 5141 hybrid_clock.cc:648] HybridClock initialized: now 1786412860279142 us; error 132 us; skew 500 ppm
I20260811 01:47:40.279711 5141 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:40 dist-test-slave-9gcw krb5kdc[3581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412860, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:40.284137 5141 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:47:40.287403 5141 webserver.cc:460] Webserver started at http://127.21.214.67:41927/ using document root <none> and password file <none>
I20260811 01:47:40.288465 5141 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:40.288707 5141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 01:47:40 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412860, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
I20260811 01:47:40.316587 5141 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.004s
I20260811 01:47:40.320360 5159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:40.322527 5141 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260811 01:47:40.322721 5141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "b6ec958527af488f930237622caa9a80"
format_stamp: "Formatted at 2026-08-11 01:47:36 on dist-test-slave-9gcw"
server_key: "3cd83f0d39356e1a5cb932ff3b06dbbd"
server_key_iv: "4b608547a65a8251d0982b6968bc3203"
server_key_version: "kuduclusterkey@0"
I20260811 01:47:40.324211 5141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:40.371850 5141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:40.379390 5141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:40.379832 5141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:40.381361 5141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:47:40.383312 5141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:47:40.383426 5141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:40.383606 5141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:47:40.383697 5141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:40.449553 5141 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:35229
I20260811 01:47:40.449635 5285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:35229 every 8 connection(s)
I20260811 01:47:40.451891 5141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:47:40.455880 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 5141
Aug 11 01:47:40 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412860, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:47:40.490165 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:40.457412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57488 (local address 127.21.214.126:44727)
0811 01:47:40.469656 (+ 12244us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:40.469663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:40.469695 (+ 32us) server_negotiation.cc:407] Connection header received
0811 01:47:40.469783 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:40.469790 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:40.469896 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:40.470046 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:40.472651 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.473855 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.475938 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.476565 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.476691 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:40.481467 (+ 4776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:40.481474 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:40.481479 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:40.481529 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:40.485810 (+ 4281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:40.486894 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:40.486902 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:40.486905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:40.487065 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:40.487778 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:40.487782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:40.487784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:40.488153 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:40.488544 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:40.489317 (+ 773us) server_negotiation.cc:299] Negotiation successful
0811 01:47:40.489651 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12061,"thread_start_us":11914,"threads_started":1}
I20260811 01:47:40.503338 5293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:40.461417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:36455 (local address 127.21.214.126:44727)
0811 01:47:40.473600 (+ 12183us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:40.473606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:40.473682 (+ 76us) server_negotiation.cc:407] Connection header received
0811 01:47:40.473738 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:40.473752 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:40.473854 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:40.473963 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:40.477557 (+ 3594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.479502 (+ 1945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.482741 (+ 3239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.483576 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.483726 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:40.494584 (+ 10858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:40.494591 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:40.494596 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:40.494643 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:40.499032 (+ 4389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:40.500257 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:40.500265 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:40.500271 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:40.500424 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:40.501033 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:40.501038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:40.501041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:40.501443 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:40.501673 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:40.502548 (+ 875us) server_negotiation.cc:299] Negotiation successful
0811 01:47:40.502807 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12037,"thread_start_us":13164,"threads_started":1}
I20260811 01:47:40.504652 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:40.460346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:44727 (local address 127.21.214.67:36455)
0811 01:47:40.470142 (+ 9796us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:40.470211 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:40.472879 (+ 2668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:40.475391 (+ 2512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:40.475405 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:40.476203 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:40.477248 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:40.477299 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.479902 (+ 2603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.479908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:40.482488 (+ 2580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:40.482507 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.484137 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.484146 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:40.484499 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:40.486009 (+ 1510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:40.486050 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:40.493123 (+ 7073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:40.499278 (+ 6155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:40.499290 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:40.499313 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:40.500041 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:40.500566 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:40.500571 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:40.500575 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:40.500849 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:40.501708 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:40.501724 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:40.502146 (+ 422us) client_negotiation.cc:769] Sending connection context
0811 01:47:40.502447 (+ 301us) client_negotiation.cc:240] Negotiation successful
0811 01:47:40.503264 (+ 817us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9402,"thread_start_us":9111,"threads_started":1}
I20260811 01:47:40.508189 5287 heartbeater.cc:347] Connected to a master server at 127.21.214.126:44727
I20260811 01:47:40.508563 5287 heartbeater.cc:464] Registering TS with master...
I20260811 01:47:40.509620 5287 heartbeater.cc:511] Master 127.21.214.126:44727 requested a full tablet report, sending...
I20260811 01:47:40.511540 4744 ts_manager.cc:194] Registered new tserver with Master: b6ec958527af488f930237622caa9a80 (127.21.214.67:35229)
I20260811 01:47:40.512652 4744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:36455
I20260811 01:47:40.522138 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 01:47:40 dist-test-slave-9gcw krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412859, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for HTTP/127.21.214.84@KRBTEST.COM
I20260811 01:47:40.566562 5293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:40.537255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57492 (local address 127.21.214.126:44727)
0811 01:47:40.537467 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:40.537474 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:40.544872 (+ 7398us) server_negotiation.cc:407] Connection header received
0811 01:47:40.545498 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:40.545505 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:40.545604 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:40.545684 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:40.548474 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.549860 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.552180 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.552922 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.553033 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:40.558260 (+ 5227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:40.558270 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:40.558275 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:40.558332 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:40.562504 (+ 4172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:40.563706 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:40.563713 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:40.563717 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:40.563910 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:40.564453 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:40.564459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:40.564463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:40.564904 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:40.565193 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:40.565884 (+ 691us) server_negotiation.cc:299] Negotiation successful
0811 01:47:40.566175 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 01:47:40.575448 4744 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:47:40.588305 4744 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:47:40.657450 5310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:40.610025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:35229 (local address 127.0.0.1:33836)
0811 01:47:40.638314 (+ 28289us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:40.638342 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:40.638537 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:40.642513 (+ 3976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:40.642519 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:40.642550 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:47:40.643043 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:40.643072 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.647612 (+ 4540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.647632 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:40.652226 (+ 4594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:40.652249 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.656106 (+ 3857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.656115 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:40.656508 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:40.656577 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 01:47:40.656729 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 01:47:40.656992 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28044,"thread_start_us":14199,"threads_started":1}
I20260811 01:47:40.658553 5308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:40.611392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33836 (local address 127.21.214.67:35229)
0811 01:47:40.638798 (+ 27406us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:40.638810 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:40.638855 (+ 45us) server_negotiation.cc:407] Connection header received
0811 01:47:40.639024 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:40.639049 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:40.639422 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:40.639726 (+ 304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:47:40.643322 (+ 3596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.647380 (+ 4058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.652545 (+ 5165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.655459 (+ 2914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.655563 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:40.655853 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:40.657948 (+ 2095us) server_negotiation.cc:299] Negotiation successful
0811 01:47:40.658148 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27160,"thread_start_us":28587,"threads_started":1}
I20260811 01:47:40.660861 5305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:40.609937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:46155 (local address 127.0.0.1:44506)
0811 01:47:40.621951 (+ 12014us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:40.622044 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:40.622331 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:40.639405 (+ 17074us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:40.639415 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:40.639449 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:47:40.639884 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:40.639898 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.653391 (+ 13493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.653397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:40.655955 (+ 2558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:40.655977 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.660031 (+ 4054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.660036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:40.660167 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:40.660193 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 01:47:40.660309 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 01:47:40.660470 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11662,"thread_start_us":30389,"threads_started":1}
I20260811 01:47:40.661859 5307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:40.610732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44506 (local address 127.21.214.65:46155)
0811 01:47:40.623107 (+ 12375us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:40.623117 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:40.623151 (+ 34us) server_negotiation.cc:407] Connection header received
0811 01:47:40.623264 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:40.623273 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:40.623606 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:40.623981 (+ 375us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:47:40.641072 (+ 17091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.643071 (+ 1999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.656235 (+ 13164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.659391 (+ 3156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.659598 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:40.659892 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:40.661259 (+ 1367us) server_negotiation.cc:299] Negotiation successful
0811 01:47:40.661473 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12186,"thread_start_us":12011,"threads_started":1}
I20260811 01:47:40.664016 5212 tablet_service.cc:1467] Processing CreateTablet for tablet 2696e9a2f03b46388cc7e901f0b5146e (DEFAULT_TABLE table=test-table [id=d577e6b0fc1845189d50d343042c5233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:47:40.667263 5306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:40.631924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:42629 (local address 127.0.0.1:39874)
0811 01:47:40.640510 (+ 8586us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:40.640549 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:40.640663 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:40.657331 (+ 16668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:40.657337 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:40.657368 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:47:40.657750 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:40.657762 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.661334 (+ 3572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.661339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:40.663426 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:40.663443 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.666395 (+ 2952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.666402 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:40.666583 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:40.666616 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 01:47:40.666747 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 01:47:40.666902 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8389,"thread_start_us":6621,"threads_started":1}
I20260811 01:47:40.666848 4895 tablet_service.cc:1467] Processing CreateTablet for tablet 2696e9a2f03b46388cc7e901f0b5146e (DEFAULT_TABLE table=test-table [id=d577e6b0fc1845189d50d343042c5233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:47:40.667688 5212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2696e9a2f03b46388cc7e901f0b5146e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:47:40.668995 4895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2696e9a2f03b46388cc7e901f0b5146e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:47:40.674261 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:40.625916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39874 (local address 127.21.214.66:42629)
0811 01:47:40.643773 (+ 17857us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:40.643785 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:40.643840 (+ 55us) server_negotiation.cc:407] Connection header received
0811 01:47:40.656346 (+ 12506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:40.656473 (+ 127us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:40.656793 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:40.657047 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:47:40.657980 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.661092 (+ 3112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.664216 (+ 3124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.665846 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.665990 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:40.666225 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:40.673534 (+ 7309us) server_negotiation.cc:299] Negotiation successful
0811 01:47:40.673897 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17556,"thread_start_us":27111,"threads_started":1}
I20260811 01:47:40.679065 5065 tablet_service.cc:1467] Processing CreateTablet for tablet 2696e9a2f03b46388cc7e901f0b5146e (DEFAULT_TABLE table=test-table [id=d577e6b0fc1845189d50d343042c5233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:47:40.680989 5065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2696e9a2f03b46388cc7e901f0b5146e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:47:40.695214 5311 tablet_bootstrap.cc:492] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4: Bootstrap starting.
I20260811 01:47:40.700301 5137 heartbeater.cc:503] Master 127.21.214.126:44727 was elected leader, sending a full tablet report...
I20260811 01:47:40.703866 5311 tablet_bootstrap.cc:654] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4: Neither blocks nor log segments found. Creating new log.
I20260811 01:47:40.705696 5311 log.cc:824] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4: Log is configured to *not* fsync() on all Append() calls
I20260811 01:47:40.712105 5314 tablet_bootstrap.cc:492] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80: Bootstrap starting.
I20260811 01:47:40.716871 5314 tablet_bootstrap.cc:654] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80: Neither blocks nor log segments found. Creating new log.
I20260811 01:47:40.725356 5315 tablet_bootstrap.cc:492] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82: Bootstrap starting.
I20260811 01:47:40.728101 5311 tablet_bootstrap.cc:492] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4: No bootstrap required, opened a new log
I20260811 01:47:40.728513 5311 ts_tablet_manager.cc:1397] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4: Time spent bootstrapping tablet: real 0.034s user 0.010s sys 0.011s
I20260811 01:47:40.728904 5314 log.cc:824] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80: Log is configured to *not* fsync() on all Append() calls
I20260811 01:47:40.732651 5315 tablet_bootstrap.cc:654] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82: Neither blocks nor log segments found. Creating new log.
I20260811 01:47:40.736657 5314 tablet_bootstrap.cc:492] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80: No bootstrap required, opened a new log
I20260811 01:47:40.737003 5314 ts_tablet_manager.cc:1397] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.007s
I20260811 01:47:40.740823 5311 raft_consensus.cc:348] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3676f980fd014d92a8d93ac0764bb4d4" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 46155 } } peers { permanent_uuid: "b6ec958527af488f930237622caa9a80" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35229 } } peers { permanent_uuid: "b485ed768d0a40d7bf2468232a394a82" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42629 } }
I20260811 01:47:40.741607 5311 raft_consensus.cc:374] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:47:40.741780 5311 raft_consensus.cc:729] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3676f980fd014d92a8d93ac0764bb4d4, State: Initialized, Role: FOLLOWER
I20260811 01:47:40.742303 5315 log.cc:824] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82: Log is configured to *not* fsync() on all Append() calls
I20260811 01:47:40.742929 5311 consensus_queue.cc:260] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3676f980fd014d92a8d93ac0764bb4d4" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 46155 } } peers { permanent_uuid: "b6ec958527af488f930237622caa9a80" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35229 } } peers { permanent_uuid: "b485ed768d0a40d7bf2468232a394a82" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42629 } }
I20260811 01:47:40.748884 5311 ts_tablet_manager.cc:1428] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4: Time spent starting tablet: real 0.020s user 0.011s sys 0.006s
I20260811 01:47:40.749387 4978 heartbeater.cc:503] Master 127.21.214.126:44727 was elected leader, sending a full tablet report...
I20260811 01:47:40.753410 5314 raft_consensus.cc:348] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3676f980fd014d92a8d93ac0764bb4d4" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 46155 } } peers { permanent_uuid: "b6ec958527af488f930237622caa9a80" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35229 } } peers { permanent_uuid: "b485ed768d0a40d7bf2468232a394a82" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42629 } }
I20260811 01:47:40.754074 5314 raft_consensus.cc:374] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:47:40.754197 5314 raft_consensus.cc:729] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ec958527af488f930237622caa9a80, State: Initialized, Role: FOLLOWER
I20260811 01:47:40.755074 5314 consensus_queue.cc:260] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3676f980fd014d92a8d93ac0764bb4d4" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 46155 } } peers { permanent_uuid: "b6ec958527af488f930237622caa9a80" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35229 } } peers { permanent_uuid: "b485ed768d0a40d7bf2468232a394a82" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42629 } }
I20260811 01:47:40.760013 5287 heartbeater.cc:503] Master 127.21.214.126:44727 was elected leader, sending a full tablet report...
I20260811 01:47:40.760205 5315 tablet_bootstrap.cc:492] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82: No bootstrap required, opened a new log
I20260811 01:47:40.760648 5315 ts_tablet_manager.cc:1397] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82: Time spent bootstrapping tablet: real 0.036s user 0.010s sys 0.000s
I20260811 01:47:40.768523 5314 ts_tablet_manager.cc:1428] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80: Time spent starting tablet: real 0.031s user 0.008s sys 0.011s
I20260811 01:47:40.773286 5315 raft_consensus.cc:348] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3676f980fd014d92a8d93ac0764bb4d4" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 46155 } } peers { permanent_uuid: "b6ec958527af488f930237622caa9a80" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35229 } } peers { permanent_uuid: "b485ed768d0a40d7bf2468232a394a82" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42629 } }
I20260811 01:47:40.773871 5315 raft_consensus.cc:374] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:47:40.773985 5315 raft_consensus.cc:729] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b485ed768d0a40d7bf2468232a394a82, State: Initialized, Role: FOLLOWER
I20260811 01:47:40.774878 5315 consensus_queue.cc:260] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3676f980fd014d92a8d93ac0764bb4d4" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 46155 } } peers { permanent_uuid: "b6ec958527af488f930237622caa9a80" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35229 } } peers { permanent_uuid: "b485ed768d0a40d7bf2468232a394a82" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42629 } }
I20260811 01:47:40.781684 5315 ts_tablet_manager.cc:1428] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82: Time spent starting tablet: real 0.021s user 0.016s sys 0.007s
I20260811 01:47:40.869336 5320 raft_consensus.cc:482] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:47:40.869714 5320 raft_consensus.cc:504] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3676f980fd014d92a8d93ac0764bb4d4" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 46155 } } peers { permanent_uuid: "b6ec958527af488f930237622caa9a80" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35229 } } peers { permanent_uuid: "b485ed768d0a40d7bf2468232a394a82" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42629 } }
I20260811 01:47:40.872184 5320 leader_election.cc:288] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3676f980fd014d92a8d93ac0764bb4d4 (127.21.214.65:46155), b6ec958527af488f930237622caa9a80 (127.21.214.67:35229)
I20260811 01:47:40.881764 5307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:40.873062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:38465 (local address 127.21.214.65:46155)
0811 01:47:40.873282 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:40.873290 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:40.874294 (+ 1004us) server_negotiation.cc:407] Connection header received
0811 01:47:40.874502 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:40.874508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:40.874605 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:40.874736 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:47:40.875465 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.876245 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.879139 (+ 2894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.880399 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.880628 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:40.880738 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:40.881357 (+ 619us) server_negotiation.cc:299] Negotiation successful
0811 01:47:40.881502 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 01:47:40.881974 5308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:40.872692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:37683 (local address 127.21.214.67:35229)
0811 01:47:40.872882 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:40.872889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:40.873923 (+ 1034us) server_negotiation.cc:407] Connection header received
0811 01:47:40.874270 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:40.874276 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:40.874378 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:40.874474 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:47:40.875302 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.876482 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.879177 (+ 2695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.880698 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.880868 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:40.880953 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:40.881597 (+ 644us) server_negotiation.cc:299] Negotiation successful
0811 01:47:40.881740 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 01:47:40.882257 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:40.872597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:46155 (local address 127.21.214.66:38465)
0811 01:47:40.874152 (+ 1555us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:40.874180 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:40.874320 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:40.874779 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:40.874790 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:40.874826 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:47:40.875295 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:40.875307 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.876393 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.876398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:40.878959 (+ 2561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:40.878975 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.880893 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.880900 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:40.881104 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:40.881210 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 01:47:40.881309 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 01:47:40.881466 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1413,"mutex_wait_us":145,"thread_start_us":1229,"threads_started":1}
I20260811 01:47:40.882718 5330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:40.872475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:35229 (local address 127.21.214.66:37683)
0811 01:47:40.873794 (+ 1319us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:40.873816 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:40.874063 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:40.874496 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:40.874500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:40.874523 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:47:40.875070 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:40.875086 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.876657 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.876662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:40.878961 (+ 2299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:40.878975 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.881176 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.881186 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:40.881404 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:40.881438 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 01:47:40.882150 (+ 712us) client_negotiation.cc:240] Negotiation successful
0811 01:47:40.882315 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1162,"mutex_wait_us":65,"thread_start_us":939,"threads_started":1}
I20260811 01:47:40.883651 5236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2696e9a2f03b46388cc7e901f0b5146e" candidate_uuid: "b485ed768d0a40d7bf2468232a394a82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ec958527af488f930237622caa9a80" is_pre_election: true
I20260811 01:47:40.883651 4918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2696e9a2f03b46388cc7e901f0b5146e" candidate_uuid: "b485ed768d0a40d7bf2468232a394a82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3676f980fd014d92a8d93ac0764bb4d4" is_pre_election: true
I20260811 01:47:40.884279 5236 raft_consensus.cc:2454] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b485ed768d0a40d7bf2468232a394a82 in term 0.
I20260811 01:47:40.884277 4918 raft_consensus.cc:2454] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b485ed768d0a40d7bf2468232a394a82 in term 0.
I20260811 01:47:40.885041 5011 leader_election.cc:302] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b485ed768d0a40d7bf2468232a394a82, b6ec958527af488f930237622caa9a80; no voters:
I20260811 01:47:40.885589 5320 raft_consensus.cc:2781] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:47:40.885752 5320 raft_consensus.cc:482] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:47:40.885862 5320 raft_consensus.cc:3037] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:47:40.888310 5320 raft_consensus.cc:504] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3676f980fd014d92a8d93ac0764bb4d4" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 46155 } } peers { permanent_uuid: "b6ec958527af488f930237622caa9a80" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35229 } } peers { permanent_uuid: "b485ed768d0a40d7bf2468232a394a82" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42629 } }
I20260811 01:47:40.889009 5320 leader_election.cc:288] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [CANDIDATE]: Term 1 election: Requested vote from peers 3676f980fd014d92a8d93ac0764bb4d4 (127.21.214.65:46155), b6ec958527af488f930237622caa9a80 (127.21.214.67:35229)
I20260811 01:47:40.889569 4918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2696e9a2f03b46388cc7e901f0b5146e" candidate_uuid: "b485ed768d0a40d7bf2468232a394a82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3676f980fd014d92a8d93ac0764bb4d4"
I20260811 01:47:40.889654 5236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2696e9a2f03b46388cc7e901f0b5146e" candidate_uuid: "b485ed768d0a40d7bf2468232a394a82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ec958527af488f930237622caa9a80"
I20260811 01:47:40.889802 4918 raft_consensus.cc:3037] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:47:40.889904 5236 raft_consensus.cc:3037] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:47:40.892436 4918 raft_consensus.cc:2454] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b485ed768d0a40d7bf2468232a394a82 in term 1.
I20260811 01:47:40.892916 5013 leader_election.cc:302] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3676f980fd014d92a8d93ac0764bb4d4, b485ed768d0a40d7bf2468232a394a82; no voters:
I20260811 01:47:40.893262 5320 raft_consensus.cc:2781] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:47:40.893504 5236 raft_consensus.cc:2454] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b485ed768d0a40d7bf2468232a394a82 in term 1.
I20260811 01:47:40.893985 5320 raft_consensus.cc:686] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [term 1 LEADER]: Becoming Leader. State: Replica: b485ed768d0a40d7bf2468232a394a82, State: Running, Role: LEADER
I20260811 01:47:40.894567 5320 consensus_queue.cc:237] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3676f980fd014d92a8d93ac0764bb4d4" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 46155 } } peers { permanent_uuid: "b6ec958527af488f930237622caa9a80" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35229 } } peers { permanent_uuid: "b485ed768d0a40d7bf2468232a394a82" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42629 } }
I20260811 01:47:40.900162 4744 catalog_manager.cc:5495] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 reported cstate change: term changed from 0 to 1, leader changed from <none> to b485ed768d0a40d7bf2468232a394a82 (127.21.214.66). New cstate: current_term: 1 leader_uuid: "b485ed768d0a40d7bf2468232a394a82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3676f980fd014d92a8d93ac0764bb4d4" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 46155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ec958527af488f930237622caa9a80" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b485ed768d0a40d7bf2468232a394a82" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42629 } health_report { overall_health: HEALTHY } } }
W20260811 01:47:40.900748 5138 tablet.cc:2369] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:47:40.957198 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:40.950097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39884 (local address 127.21.214.66:42629)
0811 01:47:40.950348 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:40.950355 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:40.950377 (+ 22us) server_negotiation.cc:407] Connection header received
0811 01:47:40.950548 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:40.950553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:40.950648 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:40.950833 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:47:40.951223 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.952053 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.954206 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:40.954858 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:40.955019 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:40.955703 (+ 684us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:47:40.956377 (+ 674us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:47:40.956517 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:40.956772 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 01:47:40.956871 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 01:47:40.961781 4979 tablet.cc:2369] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:47:40.968276 4918 raft_consensus.cc:1261] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4 [term 1 FOLLOWER]: Refusing update from remote peer b485ed768d0a40d7bf2468232a394a82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:47:40.969455 5236 raft_consensus.cc:1261] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80 [term 1 FOLLOWER]: Refusing update from remote peer b485ed768d0a40d7bf2468232a394a82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:47:40.969465 5334 consensus_queue.cc:1035] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3676f980fd014d92a8d93ac0764bb4d4" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:47:40.970410 5334 consensus_queue.cc:1035] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ec958527af488f930237622caa9a80" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 01:47:40.975449 5289 tablet.cc:2369] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:47:40.985520 5339 mvcc.cc:205] Tried to move back new op lower bound from 7317147078514126848 to 7317147078228623360. Current Snapshot: MvccSnapshot[applied={T|T < 7317147078514126848}]
I20260811 01:47:40.987543 5340 mvcc.cc:205] Tried to move back new op lower bound from 7317147078514126848 to 7317147078228623360. Current Snapshot: MvccSnapshot[applied={T|T < 7317147078514126848}]
I20260811 01:47:40.997577 5341 mvcc.cc:205] Tried to move back new op lower bound from 7317147078514126848 to 7317147078228623360. Current Snapshot: MvccSnapshot[applied={T|T < 7317147078514126848}]
I20260811 01:47:41.011475 4744 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:47:41.011881 4744 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:47:41.015686 4744 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41109bf7280647e0a028ac25c76801f4: Sending DeleteTablet for 3 replicas of tablet 2696e9a2f03b46388cc7e901f0b5146e
I20260811 01:47:41.016956 5212 tablet_service.cc:1514] Processing DeleteTablet for tablet 2696e9a2f03b46388cc7e901f0b5146e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:47:41 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:33836
I20260811 01:47:41.016965 4895 tablet_service.cc:1514] Processing DeleteTablet for tablet 2696e9a2f03b46388cc7e901f0b5146e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:47:41 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:44506
I20260811 01:47:41.017980 5065 tablet_service.cc:1514] Processing DeleteTablet for tablet 2696e9a2f03b46388cc7e901f0b5146e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:47:41 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:39874
I20260811 01:47:41.018427 5348 tablet_replica.cc:330] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80: stopping tablet replica
I20260811 01:47:41.018846 5348 raft_consensus.cc:2229] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:47:41.019359 5348 raft_consensus.cc:2258] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:47:41.020545 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 4786
I20260811 01:47:41.028975 5349 tablet_replica.cc:330] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4: stopping tablet replica
I20260811 01:47:41.029816 5349 raft_consensus.cc:2229] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:47:41.030555 5349 raft_consensus.cc:2258] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:47:41.031375 5348 ts_tablet_manager.cc:1905] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:47:41.033335 5349 ts_tablet_manager.cc:1905] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:47:41.038561 5351 tablet_replica.cc:330] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82: stopping tablet replica
I20260811 01:47:41.039148 5351 raft_consensus.cc:2229] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:47:41.039222 5348 ts_tablet_manager.cc:1918] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:47:41.039444 5348 log.cc:1196] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/2696e9a2f03b46388cc7e901f0b5146e
I20260811 01:47:41.039942 5351 raft_consensus.cc:2258] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:47:41.040287 5348 ts_tablet_manager.cc:1939] T 2696e9a2f03b46388cc7e901f0b5146e P b6ec958527af488f930237622caa9a80: Deleting consensus metadata
I20260811 01:47:41.042994 4726 catalog_manager.cc:4841] TS b6ec958527af488f930237622caa9a80 (127.21.214.67:35229): tablet 2696e9a2f03b46388cc7e901f0b5146e (table test-table [id=d577e6b0fc1845189d50d343042c5233]) successfully deleted
I20260811 01:47:41.045941 5349 ts_tablet_manager.cc:1918] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:47:41.046233 5349 log.cc:1196] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/2696e9a2f03b46388cc7e901f0b5146e
I20260811 01:47:41.046933 5349 ts_tablet_manager.cc:1939] T 2696e9a2f03b46388cc7e901f0b5146e P 3676f980fd014d92a8d93ac0764bb4d4: Deleting consensus metadata
I20260811 01:47:41.049094 4730 catalog_manager.cc:4841] TS 3676f980fd014d92a8d93ac0764bb4d4 (127.21.214.65:46155): tablet 2696e9a2f03b46388cc7e901f0b5146e (table test-table [id=d577e6b0fc1845189d50d343042c5233]) successfully deleted
I20260811 01:47:41.051436 5351 ts_tablet_manager.cc:1905] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:47:41.053194 5307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:41.024279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44518 (local address 127.21.214.65:46155)
0811 01:47:41.024523 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:41.024531 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:41.031669 (+ 7138us) server_negotiation.cc:407] Connection header received
0811 01:47:41.031980 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:41.031989 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:41.032116 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:41.032224 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:41.034198 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:41.035306 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:41.038034 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:41.038799 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:41.038950 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:41.043607 (+ 4657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:41.043625 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:41.043647 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:41.043713 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:41.048032 (+ 4319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:41.049248 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:41.049258 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:41.049261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:41.049429 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:41.050484 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:41.050489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:41.050493 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:41.050974 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:41.051264 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:41.052512 (+ 1248us) server_negotiation.cc:299] Negotiation successful
0811 01:47:41.052780 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 01:47:41.054011 4973 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:41.060411 5351 ts_tablet_manager.cc:1918] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:47:41.060714 5351 log.cc:1196] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/2696e9a2f03b46388cc7e901f0b5146e
I20260811 01:47:41.061416 5351 ts_tablet_manager.cc:1939] T 2696e9a2f03b46388cc7e901f0b5146e P b485ed768d0a40d7bf2468232a394a82: Deleting consensus metadata
I20260811 01:47:41.063493 4726 catalog_manager.cc:4841] TS b485ed768d0a40d7bf2468232a394a82 (127.21.214.66:42629): tablet 2696e9a2f03b46388cc7e901f0b5146e (table test-table [id=d577e6b0fc1845189d50d343042c5233]) successfully deleted
I20260811 01:47:41.228840 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 4786
I20260811 01:47:41.257757 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 4982
I20260811 01:47:41.276041 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:41.258895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39890 (local address 127.21.214.66:42629)
0811 01:47:41.259133 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:41.259143 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:41.259182 (+ 39us) server_negotiation.cc:407] Connection header received
0811 01:47:41.259260 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:41.259266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:41.259378 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:41.259497 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:41.260988 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:41.262461 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:41.263887 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:41.264732 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:41.264881 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:41.268005 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:41.268028 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:41.268053 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:41.268126 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:41.272219 (+ 4093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:41.273282 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:41.273291 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:41.273294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:41.273466 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:41.273937 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:41.273940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:41.273942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:41.274335 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:41.274623 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:41.275196 (+ 573us) server_negotiation.cc:299] Negotiation successful
0811 01:47:41.275514 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 01:47:41.276969 5132 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:41.404695 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 4982
I20260811 01:47:41.431111 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 5141
I20260811 01:47:41.447400 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:41.432131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33850 (local address 127.21.214.67:35229)
0811 01:47:41.432819 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:41.432824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:41.432843 (+ 19us) server_negotiation.cc:407] Connection header received
0811 01:47:41.432904 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:41.432908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:41.432976 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:41.433079 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:41.434631 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:41.435394 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:41.436700 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:41.437114 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:41.437350 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:41.440247 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:41.440262 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:41.440280 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:41.440317 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:41.443747 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:41.444733 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:41.444744 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:41.444746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:41.444946 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:41.445438 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:41.445442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:41.445445 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:41.445949 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:41.446331 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:41.446646 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 01:47:41.446967 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":360,"threads_started":1}
I20260811 01:47:41.448477 5281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:41.560298 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 5141
I20260811 01:47:41.585963 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 4706
I20260811 01:47:41.587502 4778 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:41.705443 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 4706
I20260811 01:47:41.734967 22361 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 01:47:42.080989 22361 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 01:47:42.081346 22361 mini_ranger.cc:66] Stopping Ranger...
I20260811 01:47:42.428177 22361 mini_ranger.cc:68] Stopped Ranger
2026-08-11 01:47:42.428 UTC [3617] LOG: received smart shutdown request
2026-08-11 01:47:42.432 UTC [3617] LOG: background worker "logical replication launcher" (PID 3626) exited with exit code 1
2026-08-11 01:47:42.432 UTC [3621] LOG: shutting down
2026-08-11 01:47:42.645 UTC [3617] LOG: database system is shut down
2026-08-11T01:47:42Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (59932 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:47:42 dist-test-slave-9gcw krb5kdc[5366](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:47:42 dist-test-slave-9gcw krb5kdc[5366](info): set up 2 sockets
Aug 11 01:47:42 dist-test-slave-9gcw krb5kdc[5366](info): commencing operation
krb5kdc: starting...
W20260811 01:47:44.028170 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:47:44 dist-test-slave-9gcw krb5kdc[5366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:47:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:47:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:47:44.337998 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:33871
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:36319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:33871,127.21.214.125:36529,127.21.214.124:35673
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:47:44.738341 5383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:44.738663 5383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:44.738883 5383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:44.747177 5383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:47:44.747380 5383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:44.747502 5383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:44.747634 5383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:47:44.747709 5383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:47:44.760215 5383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:36319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:33871,127.21.214.125:36529,127.21.214.124:35673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:33871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:44.762039 5383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:44.763600 5383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:44.778319 5389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:44.778368 5390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:44.778419 5392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:44.779912 5383 server_base.cc:1034] running on GCE node
I20260811 01:47:44.780870 5383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:44.782649 5383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:44.783936 5383 hybrid_clock.cc:648] HybridClock initialized: now 1786412864783889 us; error 42 us; skew 500 ppm
I20260811 01:47:44.784366 5383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:44 dist-test-slave-9gcw krb5kdc[5366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412864, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:44.791772 5383 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:47:44.793664 5383 webserver.cc:460] Webserver started at http://127.21.214.126:35107/ using document root <none> and password file <none>
I20260811 01:47:44.794538 5383 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:44.794746 5383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:44.795114 5383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:47:44.798599 5383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "33e29b6ee36a41278c202a3c5bd95079"
format_stamp: "Formatted at 2026-08-11 01:47:44 on dist-test-slave-9gcw"
server_key: "73c9fee3509d9a955674dd6663bdcbf7"
server_key_iv: "6069f383082ff95dac397004578ce76d"
server_key_version: "encryptionkey@0"
I20260811 01:47:44.799494 5383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "33e29b6ee36a41278c202a3c5bd95079"
format_stamp: "Formatted at 2026-08-11 01:47:44 on dist-test-slave-9gcw"
server_key: "73c9fee3509d9a955674dd6663bdcbf7"
server_key_iv: "6069f383082ff95dac397004578ce76d"
server_key_version: "encryptionkey@0"
I20260811 01:47:44.805637 5383 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 01:47:44.809875 5400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:44.811367 5383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:47:44.811581 5383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "33e29b6ee36a41278c202a3c5bd95079"
format_stamp: "Formatted at 2026-08-11 01:47:44 on dist-test-slave-9gcw"
server_key: "73c9fee3509d9a955674dd6663bdcbf7"
server_key_iv: "6069f383082ff95dac397004578ce76d"
server_key_version: "encryptionkey@0"
I20260811 01:47:44.811890 5383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:44.829617 5383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:44.834503 5383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:44.834887 5383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:44.859244 5383 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:33871
I20260811 01:47:44.859301 5460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:33871 every 8 connection(s)
I20260811 01:47:44.860781 5383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:47:44.865041 5462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:47:44.869704 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 5383
I20260811 01:47:44.870023 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:47:44.870748 22361 external_mini_cluster.cc:1428] Setting key 59e3d4c97ab7b0bf7c5ef74c4997e1dd
I20260811 01:47:44.872504 5462 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:47:44 dist-test-slave-9gcw krb5kdc[5366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412864, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.214.125@KRBTEST.COM; defaulting to no policy
I20260811 01:47:44.899479 5466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:44.877959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59812 (local address 127.21.214.126:33871)
0811 01:47:44.878951 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:44.878961 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:44.879029 (+ 68us) server_negotiation.cc:407] Connection header received
0811 01:47:44.880450 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:44.880456 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:44.880727 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:44.880958 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:44.882403 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:44.883918 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:44.885953 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:44.886599 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:44.886830 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:44.891180 (+ 4350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:44.891189 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:44.891204 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:44.891236 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:44.894754 (+ 3518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:44.895661 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:44.895677 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:44.895679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:44.895838 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:44.896294 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:44.896306 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:44.896309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:44.896905 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:44.897201 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:44.898005 (+ 804us) server_negotiation.cc:299] Negotiation successful
0811 01:47:44.898312 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":777,"thread_start_us":559,"threads_started":1}
I20260811 01:47:44.899575 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:44.876324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:33871 (local address 127.0.0.1:59812)
0811 01:47:44.877367 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:44.877422 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:44.879979 (+ 2557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:44.880973 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:44.880980 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:44.881778 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:44.882187 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:44.882206 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:44.884310 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:44.884315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:44.885785 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:44.885799 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:44.886955 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:44.886964 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:44.887248 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:44.888102 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:44.888128 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:44.890878 (+ 2750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:44.894952 (+ 4074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:44.894969 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:44.894984 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:44.895475 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:44.895966 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:44.895974 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:44.895977 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:44.896164 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:44.897119 (+ 955us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:44.897132 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:44.897410 (+ 278us) client_negotiation.cc:769] Sending connection context
0811 01:47:44.897643 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 01:47:44.898173 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":438,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.125@KRBTEST.COM" created.
I20260811 01:47:44.903467 5462 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:47:44.904731 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:44.904198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:36529 (local address 127.0.0.1:50274)
0811 01:47:44.904417 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:44.904539 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:36529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 01:47:44.905035 5404 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.214.125:36529: connect: Connection refused (error 111)
I20260811 01:47:44.907014 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:44.906553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:36529 (local address 127.0.0.1:50286)
0811 01:47:44.906733 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:44.906856 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:36529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 01:47:44.907234 5462 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.125:36529: Network error: Client connection negotiation failed: client connection to 127.21.214.125:36529: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.125.keytab.
Entry for principal kudu/127.21.214.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.125.keytab.
I20260811 01:47:44.942879 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.125:36529
--webserver_interface=127.21.214.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.125.keytab
--principal=kudu/127.21.214.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:36319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:33871,127.21.214.125:36529,127.21.214.124:35673
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:47:44.956686 5462 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } attempt: 1
I20260811 01:47:44.958033 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:44.957547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:36529 (local address 127.0.0.1:50298)
0811 01:47:44.957759 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:44.957884 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:36529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 01:47:44.959327 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:44.958959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:36529 (local address 127.0.0.1:50304)
0811 01:47:44.959102 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:44.959175 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:36529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 01:47:44.959586 5462 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.125:36529: Network error: Client connection negotiation failed: client connection to 127.21.214.125:36529: connect: Connection refused (error 111)
I20260811 01:47:45.027913 5462 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } attempt: 2
I20260811 01:47:45.029385 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.028768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:36529 (local address 127.0.0.1:50314)
0811 01:47:45.029041 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.029175 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:36529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 01:47:45.030526 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.030141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:36529 (local address 127.0.0.1:50330)
0811 01:47:45.030308 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.030399 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:36529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 01:47:45.030831 5462 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.125:36529: Network error: Client connection negotiation failed: client connection to 127.21.214.125:36529: connect: Connection refused (error 111)
I20260811 01:47:45.122123 5462 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } attempt: 3
I20260811 01:47:45.123585 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.123087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:36529 (local address 127.0.0.1:50346)
0811 01:47:45.123338 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.123444 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:36529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260811 01:47:45.124872 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.124441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:36529 (local address 127.0.0.1:50358)
0811 01:47:45.124603 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.124694 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:36529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 01:47:45.125129 5462 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.125:36529: Network error: Client connection negotiation failed: client connection to 127.21.214.125:36529: connect: Connection refused (error 111)
I20260811 01:47:45.268496 5462 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } attempt: 4
I20260811 01:47:45.269989 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.269398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:36529 (local address 127.0.0.1:50362)
0811 01:47:45.269738 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.269844 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:36529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189}
I20260811 01:47:45.271211 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.270849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:36529 (local address 127.0.0.1:50366)
0811 01:47:45.271001 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.271091 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:36529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 01:47:45.271510 5462 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.125:36529: Network error: Client connection negotiation failed: client connection to 127.21.214.125:36529: connect: Connection refused (error 111)
W20260811 01:47:45.344718 5470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:45.345047 5470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:45.345252 5470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:45.352907 5470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:47:45.353039 5470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:45.353197 5470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:45.353294 5470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:47:45.353382 5470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:47:45.364701 5470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:36319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:33871,127.21.214.125:36529,127.21.214.124:35673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.125.keytab
--principal=kudu/127.21.214.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.125:36529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.214.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:45.366284 5470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:45.367822 5470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:45.381732 5482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:45.381771 5483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:45.381695 5485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:45.382617 5470 server_base.cc:1034] running on GCE node
I20260811 01:47:45.383548 5470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:45.385123 5470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:45.386430 5470 hybrid_clock.cc:648] HybridClock initialized: now 1786412865386387 us; error 44 us; skew 500 ppm
I20260811 01:47:45.386839 5470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:45 dist-test-slave-9gcw krb5kdc[5366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412865, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:45.390532 5470 init.cc:377] Logged in from keytab as kudu/127.21.214.125@KRBTEST.COM (short username kudu)
I20260811 01:47:45.392464 5470 webserver.cc:460] Webserver started at http://127.21.214.125:36545/ using document root <none> and password file <none>
I20260811 01:47:45.393373 5470 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:45.393528 5470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:45.393887 5470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:47:45.397318 5470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-1/data/instance:
uuid: "d10b8ccfd08f45d2a8a433ee1de489d1"
format_stamp: "Formatted at 2026-08-11 01:47:45 on dist-test-slave-9gcw"
server_key: "d09d2ea7f95b1ac663b300a8f9b3ffab"
server_key_iv: "e3e136dd84afa3bb5cde89fb0594990f"
server_key_version: "encryptionkey@0"
I20260811 01:47:45.398315 5470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-1/wal/instance:
uuid: "d10b8ccfd08f45d2a8a433ee1de489d1"
format_stamp: "Formatted at 2026-08-11 01:47:45 on dist-test-slave-9gcw"
server_key: "d09d2ea7f95b1ac663b300a8f9b3ffab"
server_key_iv: "e3e136dd84afa3bb5cde89fb0594990f"
server_key_version: "encryptionkey@0"
I20260811 01:47:45.404286 5470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 01:47:45.408106 5492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:45.409729 5470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:47:45.409930 5470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-1/wal
uuid: "d10b8ccfd08f45d2a8a433ee1de489d1"
format_stamp: "Formatted at 2026-08-11 01:47:45 on dist-test-slave-9gcw"
server_key: "d09d2ea7f95b1ac663b300a8f9b3ffab"
server_key_iv: "e3e136dd84afa3bb5cde89fb0594990f"
server_key_version: "encryptionkey@0"
I20260811 01:47:45.410142 5470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:45.422819 5470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:45.427906 5470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:45.428233 5470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:45.451087 5470 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.125:36529
I20260811 01:47:45.451133 5546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.125:36529 every 8 connection(s)
I20260811 01:47:45.452626 5470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-1/data/info.pb
I20260811 01:47:45.454689 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 5470
I20260811 01:47:45.454998 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-1/wal/instance
I20260811 01:47:45.455596 22361 external_mini_cluster.cc:1428] Setting key fab7048dd37130ec49992a82d399d581
I20260811 01:47:45.458382 5547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:47:45.469269 5547 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.214.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.124@KRBTEST.COM" created.
Aug 11 01:47:45 dist-test-slave-9gcw krb5kdc[5366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412865, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.125@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:47:45.497790 5550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.472487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59814 (local address 127.21.214.126:33871)
0811 01:47:45.473138 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:45.473203 (+ 65us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:45.474503 (+ 1300us) server_negotiation.cc:407] Connection header received
0811 01:47:45.476812 (+ 2309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:45.476817 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:45.476928 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:45.477076 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:45.479051 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:45.480773 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:45.482580 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:45.483022 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:45.483179 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:45.489461 (+ 6282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:45.489472 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:45.489480 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:45.489543 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:45.493835 (+ 4292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:45.494890 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:45.494900 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:45.494906 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:45.495083 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:45.495565 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:45.495573 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:45.495579 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:45.496040 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:45.496207 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:45.496962 (+ 755us) server_negotiation.cc:299] Negotiation successful
0811 01:47:45.497270 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":404,"threads_started":1}
I20260811 01:47:45.498637 5551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.473104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:33871 (local address 127.0.0.1:59814)
0811 01:47:45.474290 (+ 1186us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.474343 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:45.476455 (+ 2112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:45.477443 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:45.477453 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:45.478069 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:45.478820 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:45.478848 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:45.480977 (+ 2129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:45.480981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:45.482409 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:45.482424 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:45.483294 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:45.483302 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:45.483549 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:45.484361 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:45.484387 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:45.489040 (+ 4653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:45.494093 (+ 5053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:45.494104 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:45.494122 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:45.494687 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:45.495210 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:45.495213 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:45.495215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:45.495432 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:45.496171 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:45.496178 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:45.496556 (+ 378us) client_negotiation.cc:769] Sending connection context
0811 01:47:45.496800 (+ 244us) client_negotiation.cc:240] Negotiation successful
0811 01:47:45.497347 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":839,"thread_start_us":348,"threads_started":1}
I20260811 01:47:45.500814 5547 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:47:45 dist-test-slave-9gcw krb5kdc[5366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412865, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.125@KRBTEST.COM for kudu/127.21.214.125@KRBTEST.COM
I20260811 01:47:45.519941 5551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.501539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:36529 (local address 127.0.0.1:50372)
0811 01:47:45.501720 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.501755 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:45.501864 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:45.503240 (+ 1376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:45.503244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:45.503712 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:45.503933 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:45.503943 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:45.506111 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:45.506115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:45.507010 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:45.507023 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:45.508404 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:45.508409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:45.508563 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:45.509298 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:45.509333 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:45.511814 (+ 2481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:45.517030 (+ 5216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:45.517035 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:45.517037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:45.517528 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:45.518115 (+ 587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:45.518119 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:45.518122 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:45.518285 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:45.518950 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:45.518954 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:45.519169 (+ 215us) client_negotiation.cc:769] Sending connection context
0811 01:47:45.519301 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 01:47:45.519520 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 01:47:45.520031 5553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.501821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50372 (local address 127.21.214.125:36529)
0811 01:47:45.502586 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:45.502597 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:45.502632 (+ 35us) server_negotiation.cc:407] Connection header received
0811 01:47:45.502693 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:45.502703 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:45.502973 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:45.503137 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:45.504115 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:45.505944 (+ 1829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:45.507164 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:45.508207 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:45.508359 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:45.512058 (+ 3699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:45.512077 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:45.512102 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:45.512152 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:45.516856 (+ 4704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:45.517727 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:45.517736 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:45.517740 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:45.517968 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:45.518421 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:45.518424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:45.518426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:45.518803 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:45.518986 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:45.519455 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 01:47:45.519656 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":304,"threads_started":1}
I20260811 01:47:45.522811 5547 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:47:45.524248 5551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.523722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60004)
0811 01:47:45.523943 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.524053 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260811 01:47:45.524657 5493 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
I20260811 01:47:45.526401 5551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.525949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60020)
0811 01:47:45.526199 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.526293 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 01:47:45.526633 5547 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:35673: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.124.keytab.
Entry for principal kudu/127.21.214.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.124.keytab.
I20260811 01:47:45.529706 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.124:35673
--webserver_interface=127.21.214.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.124.keytab
--principal=kudu/127.21.214.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:36319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:33871,127.21.214.125:36529,127.21.214.124:35673
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:47:45.570845 5462 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } attempt: 5
I20260811 01:47:45.575984 5547 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } attempt: 1
I20260811 01:47:45.577212 5551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.576633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60022)
0811 01:47:45.576886 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.577008 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 01:47:45.578318 5551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.577882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60038)
0811 01:47:45.578062 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.578167 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 01:47:45.578668 5547 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:35673: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Aug 11 01:47:45 dist-test-slave-9gcw krb5kdc[5366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412864, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for kudu/127.21.214.125@KRBTEST.COM
I20260811 01:47:45.586570 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.571657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:36529 (local address 127.0.0.1:50382)
0811 01:47:45.571893 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.571917 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:45.572055 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:45.572471 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:45.572476 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:45.572985 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:45.573281 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:45.573296 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:45.574344 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:45.574350 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:45.575234 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:45.575247 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:45.576053 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:45.576059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:45.576272 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:45.577003 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:45.577026 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:45.579716 (+ 2690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:45.583938 (+ 4222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:45.583943 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:45.583946 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:45.584378 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:45.584803 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:45.584806 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:45.584808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:45.584942 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:45.585727 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:45.585731 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:45.585884 (+ 153us) client_negotiation.cc:769] Sending connection context
0811 01:47:45.586007 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 01:47:45.586316 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 01:47:45.586776 5553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.571850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50382 (local address 127.21.214.125:36529)
0811 01:47:45.572066 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:45.572073 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:45.572102 (+ 29us) server_negotiation.cc:407] Connection header received
0811 01:47:45.572181 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:45.572186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:45.572292 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:45.572405 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:45.573447 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:45.574187 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:45.575418 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:45.575886 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:45.576037 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:45.579920 (+ 3883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:45.579927 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:45.579933 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:45.579977 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:45.583728 (+ 3751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:45.584511 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:45.584519 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:45.584524 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:45.584684 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:45.585063 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:45.585070 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:45.585075 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:45.585575 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:45.585753 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:45.586155 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 01:47:45.586424 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 01:47:45.587915 5462 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:47:45.589048 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.588659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60054)
0811 01:47:45.588839 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.588919 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 01:47:45.590160 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.589826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60062)
0811 01:47:45.589972 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.590058 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 01:47:45.590435 5462 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:35673: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
I20260811 01:47:45.641686 5462 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } attempt: 1
I20260811 01:47:45.642969 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.642474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60072)
0811 01:47:45.642684 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.642791 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 01:47:45.644070 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.643703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60078)
0811 01:47:45.643866 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.643964 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 01:47:45.644296 5462 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:35673: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
I20260811 01:47:45.646939 5547 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } attempt: 2
I20260811 01:47:45.648391 5551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.647784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60082)
0811 01:47:45.648045 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.648183 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 01:47:45.649611 5551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.649223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60090)
0811 01:47:45.649379 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.649468 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 01:47:45.649914 5547 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:35673: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
I20260811 01:47:45.712520 5462 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } attempt: 2
I20260811 01:47:45.713824 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.713306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60092)
0811 01:47:45.713567 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.713656 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 01:47:45.714923 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.714584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60096)
0811 01:47:45.714743 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.714814 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 01:47:45.715179 5462 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:35673: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
I20260811 01:47:45.741204 5547 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } attempt: 3
I20260811 01:47:45.742496 5551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.741928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60104)
0811 01:47:45.742171 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.742291 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 01:47:45.743749 5551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.743157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60120)
0811 01:47:45.743435 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.743547 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260811 01:47:45.744031 5547 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:35673: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
I20260811 01:47:45.821506 5462 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } attempt: 3
I20260811 01:47:45.822677 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.822208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60134)
0811 01:47:45.822413 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.822513 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 01:47:45.824025 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.823520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60146)
0811 01:47:45.823838 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.823917 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
W20260811 01:47:45.824321 5462 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:35673: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
I20260811 01:47:45.887318 5547 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } attempt: 4
I20260811 01:47:45.888698 5551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.888079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60160)
0811 01:47:45.888361 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.888485 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 01:47:45.890182 5551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:45.889689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60170)
0811 01:47:45.889887 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:45.890004 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 01:47:45.890421 5547 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:35673: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
W20260811 01:47:45.920418 5555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:45.920744 5555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:45.920836 5555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:45.928996 5555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:47:45.929191 5555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:45.929334 5555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:45.929426 5555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:47:45.929468 5555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:47:45.940050 5555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:36319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:33871,127.21.214.125:36529,127.21.214.124:35673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.124.keytab
--principal=kudu/127.21.214.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.124:35673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.214.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:45.941684 5555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:45.943037 5555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:45.955832 5562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:45.955911 5567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:45.955873 5563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:45.957026 5555 server_base.cc:1034] running on GCE node
I20260811 01:47:45.957969 5555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:45.959528 5555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:45.960748 5555 hybrid_clock.cc:648] HybridClock initialized: now 1786412865960651 us; error 79 us; skew 500 ppm
I20260811 01:47:45.961131 5555 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:45 dist-test-slave-9gcw krb5kdc[5366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412865, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:45.964655 5555 init.cc:377] Logged in from keytab as kudu/127.21.214.124@KRBTEST.COM (short username kudu)
I20260811 01:47:45.966272 5555 webserver.cc:460] Webserver started at http://127.21.214.124:40979/ using document root <none> and password file <none>
I20260811 01:47:45.967042 5555 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:45.967183 5555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:45.967553 5555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:47:45.970939 5555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-2/data/instance:
uuid: "0b0c904b18774a24887999107b11705c"
format_stamp: "Formatted at 2026-08-11 01:47:45 on dist-test-slave-9gcw"
server_key: "46e4949e247d45a4c949c0bf2463ec5f"
server_key_iv: "b7fdbdbeb290a8777f5e10dc882498de"
server_key_version: "encryptionkey@0"
I20260811 01:47:45.972010 5555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-2/wal/instance:
uuid: "0b0c904b18774a24887999107b11705c"
format_stamp: "Formatted at 2026-08-11 01:47:45 on dist-test-slave-9gcw"
server_key: "46e4949e247d45a4c949c0bf2463ec5f"
server_key_iv: "b7fdbdbeb290a8777f5e10dc882498de"
server_key_version: "encryptionkey@0"
I20260811 01:47:45.978118 5555 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 01:47:45.982043 5574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:45.983564 5555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:47:45.983760 5555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-2/wal
uuid: "0b0c904b18774a24887999107b11705c"
format_stamp: "Formatted at 2026-08-11 01:47:45 on dist-test-slave-9gcw"
server_key: "46e4949e247d45a4c949c0bf2463ec5f"
server_key_iv: "b7fdbdbeb290a8777f5e10dc882498de"
server_key_version: "encryptionkey@0"
I20260811 01:47:45.983999 5555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:46.001550 5462 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } attempt: 4
I20260811 01:47:46.002764 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:46.002290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60186)
0811 01:47:46.002506 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:46.002609 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 01:47:46.003881 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:46.003519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60194)
0811 01:47:46.003681 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:46.003756 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 01:47:46.004237 5462 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:35673: Network error: Client connection negotiation failed: client connection to 127.21.214.124:35673: connect: Connection refused (error 111)
I20260811 01:47:46.011348 5555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:46.016768 5555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:46.017208 5555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:46.040297 5555 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.124:35673
I20260811 01:47:46.040349 5633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.124:35673 every 8 connection(s)
I20260811 01:47:46.041941 5555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-2/data/info.pb
I20260811 01:47:46.046104 5634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:47:46.050324 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 5555
I20260811 01:47:46.050546 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786412636131983-22361-0/minicluster-data/master-2/wal/instance
I20260811 01:47:46.051162 22361 external_mini_cluster.cc:1428] Setting key 6ccebeb40e576f8ee363ea950e49c675
I20260811 01:47:46.054786 5634 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:47:46 dist-test-slave-9gcw krb5kdc[5366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.124@KRBTEST.COM
Aug 11 01:47:46 dist-test-slave-9gcw krb5kdc[5366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412865, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.124@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:47:46.076520 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:46.059622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59826 (local address 127.21.214.126:33871)
0811 01:47:46.060250 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:46.060257 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:46.060841 (+ 584us) server_negotiation.cc:407] Connection header received
0811 01:47:46.061005 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:46.061009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:46.061085 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:46.061253 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:46.062678 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.063611 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.065421 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.065849 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.066026 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:46.070303 (+ 4277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:46.070308 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:46.070312 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:46.070349 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:46.073504 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.074259 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.074264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.074269 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.074407 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.074794 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.074801 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.074806 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.075070 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:46.075254 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:46.075824 (+ 570us) server_negotiation.cc:299] Negotiation successful
0811 01:47:46.076046 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":259,"threads_started":1}
I20260811 01:47:46.077418 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:46.059665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:33871 (local address 127.0.0.1:59826)
0811 01:47:46.060682 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:46.060728 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:46.060886 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:46.061319 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:46.061326 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:46.062231 (+ 905us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:46.062514 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:46.062522 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.063778 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.063786 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:46.065273 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:46.065286 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.066094 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.066101 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:46.066316 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:46.066996 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:46.067022 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:46.070055 (+ 3033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:46.073659 (+ 3604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:46.073671 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:46.073679 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:46.074121 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:46.074526 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:46.074531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:46.074535 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:46.074670 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:46.075206 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:46.075214 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:46.075518 (+ 304us) client_negotiation.cc:769] Sending connection context
0811 01:47:46.075724 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 01:47:46.076254 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":161,"spinlock_wait_cycles":18304,"thread_start_us":580,"threads_started":1}
I20260811 01:47:46.077437 5639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:46.053427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60204 (local address 127.21.214.124:35673)
0811 01:47:46.054328 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:46.054340 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:46.054384 (+ 44us) server_negotiation.cc:407] Connection header received
0811 01:47:46.056312 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:46.056341 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:46.056793 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:46.057239 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:46.059241 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.061619 (+ 2378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.063330 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.064292 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.064491 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:46.069385 (+ 4894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:46.069398 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:46.069413 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:46.069447 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:46.072594 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.073457 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.073465 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.073473 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.073652 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.074138 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.074142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.074152 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.074747 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:46.075065 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:46.075532 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 01:47:46.076088 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":436,"threads_started":1}
I20260811 01:47:46.079255 5634 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:47:46 dist-test-slave-9gcw krb5kdc[5366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412865, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.124@KRBTEST.COM for kudu/127.21.214.125@KRBTEST.COM
Aug 11 01:47:46 dist-test-slave-9gcw krb5kdc[5366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:47:46.095432 5553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:46.080136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50398 (local address 127.21.214.125:36529)
0811 01:47:46.080306 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:46.080313 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:46.080416 (+ 103us) server_negotiation.cc:407] Connection header received
0811 01:47:46.080622 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:46.080628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:46.080713 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:46.080842 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:46.081910 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.083042 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.084684 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.085384 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.085516 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:46.089263 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:46.089270 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:46.089275 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:46.089327 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:46.092478 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.093366 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.093370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.093372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.093503 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.093997 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.094000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.094002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.094331 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:46.094501 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:46.094957 (+ 456us) server_negotiation.cc:299] Negotiation successful
0811 01:47:46.095126 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 01:47:46.095564 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:46.080030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:36529 (local address 127.0.0.1:50398)
0811 01:47:46.080304 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:46.080326 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:46.080493 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:46.081245 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:46.081249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:46.081595 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:46.081749 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:46.081758 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.083191 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.083199 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:46.084544 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:46.084557 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.085527 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.085532 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:46.085701 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:46.086399 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:46.086422 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:46.088985 (+ 2563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:46.092619 (+ 3634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:46.092624 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:46.092628 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:46.093231 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:46.093647 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:46.093652 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:46.093656 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:46.093872 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:46.094463 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:46.094470 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:46.094666 (+ 196us) client_negotiation.cc:769] Sending connection context
0811 01:47:46.094868 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 01:47:46.095183 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260811 01:47:46.096612 5634 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:47:46.099474 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:46.080825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59830 (local address 127.21.214.126:33871)
0811 01:47:46.081092 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:46.081096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:46.081110 (+ 14us) server_negotiation.cc:407] Connection header received
0811 01:47:46.081210 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:46.081215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:46.081297 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:46.081407 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:46.083105 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.084232 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.086098 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.086890 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.087069 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:46.092375 (+ 5306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:46.092380 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:46.092383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:46.092418 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:46.096258 (+ 3840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.097183 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.097190 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.097196 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.097400 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.097851 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.097857 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.097862 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.098275 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:46.098543 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:46.098855 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 01:47:46.099112 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
Aug 11 01:47:46 dist-test-slave-9gcw krb5kdc[5366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412865, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.124@KRBTEST.COM for kudu/127.21.214.124@KRBTEST.COM
Aug 11 01:47:46 dist-test-slave-9gcw krb5kdc[5366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.125@KRBTEST.COM
I20260811 01:47:46.110852 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:46.097563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60220)
0811 01:47:46.097848 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:46.097866 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:46.097986 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:46.098275 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:46.098281 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:46.098832 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:46.099022 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:46.099036 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.100302 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.100307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:46.101326 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:46.101345 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.102267 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.102271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:46.102402 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:46.103052 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:46.103069 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:46.105194 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:46.108328 (+ 3134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:46.108335 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:46.108340 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:46.108689 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:46.109057 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:46.109062 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:46.109066 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:46.109306 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:46.109953 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:46.109957 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:46.110083 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 01:47:46.110256 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 01:47:46.110486 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260811 01:47:46.111094 5639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:46.097538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60220 (local address 127.21.214.124:35673)
0811 01:47:46.097724 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:46.097731 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:46.097944 (+ 213us) server_negotiation.cc:407] Connection header received
0811 01:47:46.098089 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:46.098092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:46.098149 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:46.098221 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:46.099184 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.100131 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.101486 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.102027 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.102102 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:46.105396 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:46.105400 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:46.105403 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:46.105432 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:46.108192 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.108814 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.108821 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.108825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.108944 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.109413 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.109421 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.109427 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.109812 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:46.109972 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:46.110413 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 01:47:46.110694 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 01:47:46.118018 5634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c: Bootstrap starting.
I20260811 01:47:46.118602 5553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:46.101529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50400 (local address 127.21.214.125:36529)
0811 01:47:46.101744 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:46.101751 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:46.101768 (+ 17us) server_negotiation.cc:407] Connection header received
0811 01:47:46.101828 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:46.101831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:46.101886 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:46.101990 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:46.103223 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.103970 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.105929 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.106390 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.106532 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:46.111613 (+ 5081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:46.111621 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:46.111625 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:46.111673 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:46.115425 (+ 3752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.116318 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.116324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.116327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.116536 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.117000 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.117006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.117009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.117406 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:46.117702 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:46.118014 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 01:47:46.118241 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 01:47:46.122500 5634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c: Neither blocks nor log segments found. Creating new log.
I20260811 01:47:46.123894 5634 log.cc:824] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c: Log is configured to *not* fsync() on all Append() calls
I20260811 01:47:46.124431 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:47:46.127553 5634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c: No bootstrap required, opened a new log
I20260811 01:47:46.136509 5634 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } }
I20260811 01:47:46.136950 5634 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:47:46.137097 5634 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b0c904b18774a24887999107b11705c, State: Initialized, Role: FOLLOWER
I20260811 01:47:46.138157 5634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } }
I20260811 01:47:46.142421 5650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } } }
I20260811 01:47:46.143050 5650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:47:46.144156 5634 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:47:46.161998 5639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:46.130688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60224 (local address 127.21.214.124:35673)
0811 01:47:46.130925 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:46.130931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:46.138432 (+ 7501us) server_negotiation.cc:407] Connection header received
0811 01:47:46.138711 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:46.138717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:46.138822 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:46.138921 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:46.143709 (+ 4788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.144977 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.147502 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.148237 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.148343 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:46.154327 (+ 5984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:46.154336 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:46.154343 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:46.154394 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:46.158330 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.159055 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.159061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.159065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.159262 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.159778 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.159784 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.159787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.160291 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:46.160559 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:46.161311 (+ 752us) server_negotiation.cc:299] Negotiation successful
0811 01:47:46.161557 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 01:47:46.163973 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:46.131129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59846 (local address 127.21.214.126:33871)
0811 01:47:46.131311 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:46.131318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:46.137830 (+ 6512us) server_negotiation.cc:407] Connection header received
0811 01:47:46.138746 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:46.138755 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:46.138875 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:46.138971 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:46.143085 (+ 4114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.144220 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.147359 (+ 3139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.148325 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.148459 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:46.154562 (+ 6103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:46.154573 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:46.154586 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:46.154659 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:46.160579 (+ 5920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.161566 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.161571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.161574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.161737 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.162214 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.162219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.162222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.162582 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:46.162870 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:46.163465 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 01:47:46.163712 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 01:47:46.165622 5553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:46.138118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50402 (local address 127.21.214.125:36529)
0811 01:47:46.138340 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:46.138347 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:46.143819 (+ 5472us) server_negotiation.cc:407] Connection header received
0811 01:47:46.144075 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:46.144080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:46.144169 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:46.144246 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:46.147068 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.148211 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.150487 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.151237 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.151386 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:46.155785 (+ 4399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:46.155791 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:46.155796 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:46.155840 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:46.160017 (+ 4177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.161339 (+ 1322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.161348 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.161354 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.161573 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.162135 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.162143 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.162149 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.162666 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:46.162985 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:46.164769 (+ 1784us) server_negotiation.cc:299] Negotiation successful
0811 01:47:46.165136 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260811 01:47:46.175326 5663 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 01:47:46.175519 5663 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 01:47:46.172289 5555 master_runner.cc:186] Error getting master registration for 127.21.214.126:33871: OK, instance_id { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" instance_seqno: 1786412864852106 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 01:47:46.186443 5555 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:47:46.189749 5547 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } attempt: 5
Aug 11 01:47:46 dist-test-slave-9gcw krb5kdc[5366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412865, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.125@KRBTEST.COM for kudu/127.21.214.124@KRBTEST.COM
I20260811 01:47:46.207973 5639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:46.190750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60226 (local address 127.21.214.124:35673)
0811 01:47:46.190942 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:46.190947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:46.190964 (+ 17us) server_negotiation.cc:407] Connection header received
0811 01:47:46.191013 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:46.191017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:46.191074 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:46.191169 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:46.192288 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.193313 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.195309 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.196241 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.196398 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:46.200359 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:46.200365 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:46.200369 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:46.200408 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:46.204865 (+ 4457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.205793 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.205798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.205800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.205949 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.206497 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.206501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.206504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.206870 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:46.206992 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:46.207512 (+ 520us) server_negotiation.cc:299] Negotiation successful
0811 01:47:46.207689 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 01:47:46.208360 5551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:46.190427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60226)
0811 01:47:46.190681 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:46.190709 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:46.190838 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:46.191234 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:46.191240 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:46.191861 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:46.192098 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:46.192113 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.193491 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.193498 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:46.195126 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:46.195145 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.196392 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.196399 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:46.196795 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:46.197665 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:46.197690 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:46.200127 (+ 2437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:46.205020 (+ 4893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:46.205028 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:46.205032 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:46.205658 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:46.206085 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:46.206090 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:46.206097 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:46.206344 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:46.207008 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:46.207014 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:46.207209 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 01:47:46.207433 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 01:47:46.207772 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 01:47:46.215662 5547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1: Bootstrap starting.
I20260811 01:47:46.220746 5547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1: Neither blocks nor log segments found. Creating new log.
I20260811 01:47:46.222584 5547 log.cc:824] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1: Log is configured to *not* fsync() on all Append() calls
I20260811 01:47:46.226325 5547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1: No bootstrap required, opened a new log
I20260811 01:47:46.239382 5547 raft_consensus.cc:348] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } }
I20260811 01:47:46.240041 5547 raft_consensus.cc:374] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:47:46.240193 5547 raft_consensus.cc:729] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d10b8ccfd08f45d2a8a433ee1de489d1, State: Initialized, Role: FOLLOWER
I20260811 01:47:46.240983 5547 consensus_queue.cc:260] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } }
I20260811 01:47:46.242046 5667 sys_catalog.cc:455] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } } }
I20260811 01:47:46.242420 5667 sys_catalog.cc:458] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:47:46.243703 5547 sys_catalog.cc:564] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 01:47:46.256906 5679 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 01:47:46.257177 5679 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 01:47:46.253732 5470 master_runner.cc:186] Error getting master registration for 127.21.214.126:33871: OK, instance_id { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" instance_seqno: 1786412864852106 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 01:47:46.269181 5470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 01:47:46.269845 5581 tablet.cc:2369] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:47:46.281494 5462 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } attempt: 5
Aug 11 01:47:46 dist-test-slave-9gcw krb5kdc[5366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412864, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for kudu/127.21.214.124@KRBTEST.COM
I20260811 01:47:46.303229 5639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:46.282318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60234 (local address 127.21.214.124:35673)
0811 01:47:46.282530 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:46.282536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:46.282556 (+ 20us) server_negotiation.cc:407] Connection header received
0811 01:47:46.282695 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:46.282700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:46.282791 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:46.282897 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:46.284042 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.285227 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.287368 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.288094 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.288271 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:46.292911 (+ 4640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:46.292919 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:46.292926 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:46.292975 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:46.299628 (+ 6653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.300636 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.300642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.300645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.300839 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:46.301374 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:46.301380 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:46.301383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:46.301788 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:46.301975 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:46.302473 (+ 498us) server_negotiation.cc:299] Negotiation successful
0811 01:47:46.302757 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 01:47:46.303227 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:46.282204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:35673 (local address 127.0.0.1:60234)
0811 01:47:46.282398 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:46.282424 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:46.282546 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:46.282973 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:46.282980 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:46.283593 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:46.283856 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:46.283873 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.285401 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.285408 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:46.287191 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:46.287218 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:46.288300 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:46.288307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:46.288663 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:46.289528 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:46.289555 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:46.292670 (+ 3115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:46.299812 (+ 7142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:46.299820 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:46.299825 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:46.300481 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:46.300986 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:46.300992 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:46.300997 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:46.301238 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:46.301936 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:46.301944 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:46.302157 (+ 213us) client_negotiation.cc:769] Sending connection context
0811 01:47:46.302347 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 01:47:46.302680 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 01:47:46.310906 5462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079: Bootstrap starting.
I20260811 01:47:46.317966 5462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079: Neither blocks nor log segments found. Creating new log.
I20260811 01:47:46.319646 5462 log.cc:824] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079: Log is configured to *not* fsync() on all Append() calls
I20260811 01:47:46.323652 5462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079: No bootstrap required, opened a new log
I20260811 01:47:46.335808 5462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } }
I20260811 01:47:46.336288 5462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:47:46.336416 5462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33e29b6ee36a41278c202a3c5bd95079, State: Initialized, Role: FOLLOWER
I20260811 01:47:46.337273 5462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } }
I20260811 01:47:46.341631 5682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } } }
I20260811 01:47:46.342217 5682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:47:46.342983 5462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:47:46.372334 5383 master_runner.cc:263] No leader master found from master 33e29b6ee36a41278c202a3c5bd95079
I20260811 01:47:46.372510 5383 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 01:47:46.375787 5696 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 01:47:46.376091 5696 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 01:47:46.377425 5667 raft_consensus.cc:482] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:47:46.377734 5667 raft_consensus.cc:504] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } }
I20260811 01:47:46.379518 5667 leader_election.cc:288] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33e29b6ee36a41278c202a3c5bd95079 (127.21.214.126:33871), 0b0c904b18774a24887999107b11705c (127.21.214.124:35673)
I20260811 01:47:46.381567 5434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33e29b6ee36a41278c202a3c5bd95079" is_pre_election: true
I20260811 01:47:46.382315 5434 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d10b8ccfd08f45d2a8a433ee1de489d1 in term 0.
I20260811 01:47:46.383075 5607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0c904b18774a24887999107b11705c" is_pre_election: true
I20260811 01:47:46.383757 5607 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d10b8ccfd08f45d2a8a433ee1de489d1 in term 0.
I20260811 01:47:46.385475 5493 leader_election.cc:302] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b0c904b18774a24887999107b11705c, d10b8ccfd08f45d2a8a433ee1de489d1; no voters:
I20260811 01:47:46.388401 5667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:47:46.388607 5667 raft_consensus.cc:482] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:47:46.388724 5667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:47:46.392932 5667 raft_consensus.cc:504] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } }
I20260811 01:47:46.394348 5434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33e29b6ee36a41278c202a3c5bd95079"
I20260811 01:47:46.394608 5434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:47:46.398569 5434 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d10b8ccfd08f45d2a8a433ee1de489d1 in term 1.
I20260811 01:47:46.399160 5494 leader_election.cc:302] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33e29b6ee36a41278c202a3c5bd95079, d10b8ccfd08f45d2a8a433ee1de489d1; no voters:
I20260811 01:47:46.403534 5607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0c904b18774a24887999107b11705c"
I20260811 01:47:46.403812 5607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:47:46.407750 5607 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d10b8ccfd08f45d2a8a433ee1de489d1 in term 1.
I20260811 01:47:46.415055 5667 leader_election.cc:288] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [CANDIDATE]: Term 1 election: Requested vote from peers 33e29b6ee36a41278c202a3c5bd95079 (127.21.214.126:33871), 0b0c904b18774a24887999107b11705c (127.21.214.124:35673)
I20260811 01:47:46.415375 5667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:47:46.415550 5667 raft_consensus.cc:686] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [term 1 LEADER]: Becoming Leader. State: Replica: d10b8ccfd08f45d2a8a433ee1de489d1, State: Running, Role: LEADER
I20260811 01:47:46.417403 5667 consensus_queue.cc:237] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } }
I20260811 01:47:46.429047 5698 sys_catalog.cc:455] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d10b8ccfd08f45d2a8a433ee1de489d1. Latest consensus state: current_term: 1 leader_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } } }
I20260811 01:47:46.429567 5698 sys_catalog.cc:458] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [sys.catalog]: This master's current role is: LEADER
W20260811 01:47:46.431242 5498 tablet.cc:2369] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:47:46.435292 5700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:47:46.437999 5700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:47:46.463282 5607 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c [term 1 FOLLOWER]: Refusing update from remote peer d10b8ccfd08f45d2a8a433ee1de489d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:47:46.464344 5698 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:47:46.466253 5434 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079 [term 1 FOLLOWER]: Refusing update from remote peer d10b8ccfd08f45d2a8a433ee1de489d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:47:46.467586 5698 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:47:46.503742 5682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d10b8ccfd08f45d2a8a433ee1de489d1. Latest consensus state: current_term: 1 leader_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } } }
I20260811 01:47:46.504231 5682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:47:46.504165 5650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d10b8ccfd08f45d2a8a433ee1de489d1. Latest consensus state: current_term: 1 leader_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } } }
I20260811 01:47:46.504746 5650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:47:46.508504 5702 mvcc.cc:205] Tried to move back new op lower bound from 7317147101020291072 to 7317147100857876480. Current Snapshot: MvccSnapshot[applied={T|T < 7317147101020291072}]
I20260811 01:47:46.520371 5667 sys_catalog.cc:455] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } } }
I20260811 01:47:46.520897 5667 sys_catalog.cc:458] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [sys.catalog]: This master's current role is: LEADER
I20260811 01:47:46.521296 5667 sys_catalog.cc:455] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } } }
I20260811 01:47:46.521605 5667 sys_catalog.cc:458] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1 [sys.catalog]: This master's current role is: LEADER
I20260811 01:47:46.526490 5704 mvcc.cc:205] Tried to move back new op lower bound from 7317147101020291072 to 7317147100857876480. Current Snapshot: MvccSnapshot[applied={T|T < 7317147101020291072}]
I20260811 01:47:46.528200 5700 catalog_manager.cc:1347] Generated new cluster ID: 2dadc53369c54a0ba215fa0f4bfec769
I20260811 01:47:46.528333 5700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:47:46.547865 5650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } } }
I20260811 01:47:46.548240 5650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:47:46.557598 5682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e29b6ee36a41278c202a3c5bd95079" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 33871 } } peers { permanent_uuid: "d10b8ccfd08f45d2a8a433ee1de489d1" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 36529 } } peers { permanent_uuid: "0b0c904b18774a24887999107b11705c" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 35673 } } }
I20260811 01:47:46.558065 5682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:47:46.561256 5700 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:47:46.563499 5700 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:47:46.593464 5700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d10b8ccfd08f45d2a8a433ee1de489d1: Generated new TSK 0
I20260811 01:47:46.594712 5700 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:47:47.177910 5663 catalog_manager.cc:1259] Loaded cluster ID: 2dadc53369c54a0ba215fa0f4bfec769
I20260811 01:47:47.178013 5663 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c: loading cluster ID for follower catalog manager: success
I20260811 01:47:47.180629 5663 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c: acquiring CA information for follower catalog manager: success
I20260811 01:47:47.182845 5663 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0b0c904b18774a24887999107b11705c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:47:47.378717 5696 catalog_manager.cc:1259] Loaded cluster ID: 2dadc53369c54a0ba215fa0f4bfec769
I20260811 01:47:47.378854 5696 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079: loading cluster ID for follower catalog manager: success
I20260811 01:47:47.381825 5696 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079: acquiring CA information for follower catalog manager: success
I20260811 01:47:47.383857 5696 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 33e29b6ee36a41278c202a3c5bd95079: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:47:47.758781 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 5383
I20260811 01:47:47.759992 5456 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:47.871183 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 5383
I20260811 01:47:47.897009 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 5470
I20260811 01:47:47.898353 5542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:48.010269 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 5470
I20260811 01:47:48.036861 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 5555
I20260811 01:47:48.038164 5629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:48.150115 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 5555
2026-08-11T01:47:48Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5369 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:47:48 dist-test-slave-9gcw krb5kdc[5720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:47:48 dist-test-slave-9gcw krb5kdc[5720](info): set up 2 sockets
Aug 11 01:47:48 dist-test-slave-9gcw krb5kdc[5720](info): commencing operation
krb5kdc: starting...
W20260811 01:47:49.453946 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.004s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:47:49 dist-test-slave-9gcw krb5kdc[5720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:47:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:47:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:47:49.769495 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:38489
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:36927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:38489
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:47:50.127528 5736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:50.127858 5736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:50.128042 5736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:50.135603 5736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:47:50.135751 5736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:50.135800 5736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:50.135926 5736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:47:50.135991 5736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:47:50.147069 5736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:36927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:38489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:38489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:50.148531 5736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:50.150118 5736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:50.162961 5741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:50.163061 5742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:50.163075 5744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:50.164386 5736 server_base.cc:1034] running on GCE node
I20260811 01:47:50.165402 5736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:50.167035 5736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:50.168308 5736 hybrid_clock.cc:648] HybridClock initialized: now 1786412870168238 us; error 49 us; skew 500 ppm
I20260811 01:47:50.168727 5736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:50 dist-test-slave-9gcw krb5kdc[5720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412870, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:50.176167 5736 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:47:50.178436 5736 webserver.cc:460] Webserver started at http://127.21.214.126:36441/ using document root <none> and password file <none>
I20260811 01:47:50.179448 5736 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:50.179621 5736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:50.180027 5736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:47:50.183795 5736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "dc302d2e267e4e8ea54a172bff4cf3c3"
format_stamp: "Formatted at 2026-08-11 01:47:50 on dist-test-slave-9gcw"
server_key: "29e011d63b63206f907b8e240f20b1a6"
server_key_iv: "112c1dcd7045635b1c5f984bb6bb2a1b"
server_key_version: "encryptionkey@0"
I20260811 01:47:50.184707 5736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "dc302d2e267e4e8ea54a172bff4cf3c3"
format_stamp: "Formatted at 2026-08-11 01:47:50 on dist-test-slave-9gcw"
server_key: "29e011d63b63206f907b8e240f20b1a6"
server_key_iv: "112c1dcd7045635b1c5f984bb6bb2a1b"
server_key_version: "encryptionkey@0"
I20260811 01:47:50.190918 5736 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.003s
I20260811 01:47:50.195055 5752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:50.196601 5736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 01:47:50.196972 5736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "dc302d2e267e4e8ea54a172bff4cf3c3"
format_stamp: "Formatted at 2026-08-11 01:47:50 on dist-test-slave-9gcw"
server_key: "29e011d63b63206f907b8e240f20b1a6"
server_key_iv: "112c1dcd7045635b1c5f984bb6bb2a1b"
server_key_version: "encryptionkey@0"
I20260811 01:47:50.197274 5736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:50.218497 5736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:50.224025 5736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:50.224387 5736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:50.245473 5736 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:38489
I20260811 01:47:50.245477 5811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:38489 every 8 connection(s)
I20260811 01:47:50.247080 5736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:47:50.251119 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 5736
I20260811 01:47:50.251405 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:47:50.251480 5812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:47:50.252354 22361 external_mini_cluster.cc:1428] Setting key 03ca3bfc11490a45ba51a40e250a9b8c
I20260811 01:47:50.266613 5812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3: Bootstrap starting.
Aug 11 01:47:50 dist-test-slave-9gcw krb5kdc[5720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:47:50.270108 5812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3: Neither blocks nor log segments found. Creating new log.
I20260811 01:47:50.271217 5812 log.cc:824] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3: Log is configured to *not* fsync() on all Append() calls
I20260811 01:47:50.274449 5812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3: No bootstrap required, opened a new log
I20260811 01:47:50.279960 5815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:50.255008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57036 (local address 127.21.214.126:38489)
0811 01:47:50.256027 (+ 1019us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:50.256040 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:50.256086 (+ 46us) server_negotiation.cc:407] Connection header received
0811 01:47:50.258120 (+ 2034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:50.258151 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:50.258614 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:50.259023 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:50.261247 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:50.262599 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:50.264740 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:50.265677 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:50.265847 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:50.271120 (+ 5273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:50.271130 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:50.271147 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:50.271194 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:50.274811 (+ 3617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:50.275828 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:50.275837 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:50.275844 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:50.276048 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:50.276530 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:50.276535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:50.276546 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:50.277314 (+ 768us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:50.277708 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:50.278222 (+ 514us) server_negotiation.cc:299] Negotiation successful
0811 01:47:50.278777 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":459,"threads_started":1}
I20260811 01:47:50.283515 5812 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc302d2e267e4e8ea54a172bff4cf3c3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 38489 } }
I20260811 01:47:50.283975 5812 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:47:50.284125 5812 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc302d2e267e4e8ea54a172bff4cf3c3, State: Initialized, Role: FOLLOWER
I20260811 01:47:50.284754 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:47:50.284754 5812 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc302d2e267e4e8ea54a172bff4cf3c3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 38489 } }
I20260811 01:47:50.285053 5812 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:47:50.285230 5812 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:47:50.285431 5812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:47:50.287946 5812 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc302d2e267e4e8ea54a172bff4cf3c3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 38489 } }
I20260811 01:47:50.288563 5812 leader_election.cc:302] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc302d2e267e4e8ea54a172bff4cf3c3; no voters:
I20260811 01:47:50.289263 5812 leader_election.cc:288] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:47:50.289356 5817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:47:50.290303 5817 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3 [term 1 LEADER]: Becoming Leader. State: Replica: dc302d2e267e4e8ea54a172bff4cf3c3, State: Running, Role: LEADER
I20260811 01:47:50.290843 5817 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc302d2e267e4e8ea54a172bff4cf3c3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 38489 } }
I20260811 01:47:50.291653 5812 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:47:50.294144 5820 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc302d2e267e4e8ea54a172bff4cf3c3. Latest consensus state: current_term: 1 leader_uuid: "dc302d2e267e4e8ea54a172bff4cf3c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc302d2e267e4e8ea54a172bff4cf3c3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 38489 } } }
I20260811 01:47:50.294133 5818 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc302d2e267e4e8ea54a172bff4cf3c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc302d2e267e4e8ea54a172bff4cf3c3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 38489 } } }
I20260811 01:47:50.294590 5820 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3 [sys.catalog]: This master's current role is: LEADER
I20260811 01:47:50.294618 5818 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3 [sys.catalog]: This master's current role is: LEADER
I20260811 01:47:50.296729 5825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:47:50.304080 5825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:47:50.315402 5825 catalog_manager.cc:1347] Generated new cluster ID: ec5fc55b568642189e6ba88215e0bab2
I20260811 01:47:50.315534 5825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:47:50.324450 5825 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:47:50.326473 5825 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:47:50.337435 5825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc302d2e267e4e8ea54a172bff4cf3c3: Generated new TSK 0
I20260811 01:47:50.338634 5825 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:47:50.398123 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 5736
I20260811 01:47:50.399590 5807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:50.500202 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 5736
2026-08-11T01:47:50Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2344 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:47:50 dist-test-slave-9gcw krb5kdc[5842](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:47:50 dist-test-slave-9gcw krb5kdc[5842](info): set up 2 sockets
Aug 11 01:47:50 dist-test-slave-9gcw krb5kdc[5842](info): commencing operation
krb5kdc: starting...
W20260811 01:47:51.752822 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.002s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:47:51 dist-test-slave-9gcw krb5kdc[5842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:47:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:47:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:47:52.063459 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:36311
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:36311
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:47:52.434892 5856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:52.435233 5856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:52.435359 5856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:52.443531 5856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:47:52.443681 5856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:52.443735 5856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:52.443791 5856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:47:52.443843 5856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:47:52.455084 5856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:36311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:36311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:52.456547 5856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:52.458033 5856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:52.470793 5863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:52.470916 5862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:52.470896 5865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:52.471844 5856 server_base.cc:1034] running on GCE node
I20260811 01:47:52.472683 5856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:52.474292 5856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:52.475531 5856 hybrid_clock.cc:648] HybridClock initialized: now 1786412872475483 us; error 39 us; skew 500 ppm
I20260811 01:47:52.475895 5856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:52 dist-test-slave-9gcw krb5kdc[5842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412872, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:52.479241 5856 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:47:52.480847 5856 webserver.cc:460] Webserver started at http://127.21.214.126:38411/ using document root <none> and password file <none>
I20260811 01:47:52.481655 5856 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:52.481818 5856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:52.482147 5856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:47:52.485518 5856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "4cc51b4cb4844ed8b52bb5752edd8770"
format_stamp: "Formatted at 2026-08-11 01:47:52 on dist-test-slave-9gcw"
server_key: "8f1e18d709bba2fb3e650fb36e0b3daa"
server_key_iv: "eef40a02dec53de7af4fbce3d8f76c08"
server_key_version: "encryptionkey@0"
I20260811 01:47:52.486354 5856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "4cc51b4cb4844ed8b52bb5752edd8770"
format_stamp: "Formatted at 2026-08-11 01:47:52 on dist-test-slave-9gcw"
server_key: "8f1e18d709bba2fb3e650fb36e0b3daa"
server_key_iv: "eef40a02dec53de7af4fbce3d8f76c08"
server_key_version: "encryptionkey@0"
I20260811 01:47:52.492046 5856 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 01:47:52.495990 5873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:52.497576 5856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:47:52.497781 5856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "4cc51b4cb4844ed8b52bb5752edd8770"
format_stamp: "Formatted at 2026-08-11 01:47:52 on dist-test-slave-9gcw"
server_key: "8f1e18d709bba2fb3e650fb36e0b3daa"
server_key_iv: "eef40a02dec53de7af4fbce3d8f76c08"
server_key_version: "encryptionkey@0"
I20260811 01:47:52.498013 5856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:52.528815 5856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:52.533196 5856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:52.533528 5856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:52.553889 5856 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:36311
I20260811 01:47:52.553928 5932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:36311 every 8 connection(s)
I20260811 01:47:52.555464 5856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:47:52.559381 5933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:47:52.565025 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 5856
I20260811 01:47:52.565325 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:47:52.566424 22361 external_mini_cluster.cc:1428] Setting key a53432fd239188d1144f259944211780
I20260811 01:47:52.572193 5933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770: Bootstrap starting.
I20260811 01:47:52.576556 5933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770: Neither blocks nor log segments found. Creating new log.
I20260811 01:47:52.577823 5933 log.cc:824] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770: Log is configured to *not* fsync() on all Append() calls
I20260811 01:47:52.581384 5933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770: No bootstrap required, opened a new log
Aug 11 01:47:52 dist-test-slave-9gcw krb5kdc[5842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:47:52.591463 5933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36311 } }
I20260811 01:47:52.592024 5933 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:47:52.592145 5933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cc51b4cb4844ed8b52bb5752edd8770, State: Initialized, Role: FOLLOWER
I20260811 01:47:52.592958 5933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36311 } }
I20260811 01:47:52.593343 5933 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:47:52.593532 5933 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:47:52.593783 5933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:47:52.597497 5933 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36311 } }
I20260811 01:47:52.598289 5937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:52.569004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52442 (local address 127.21.214.126:36311)
0811 01:47:52.570201 (+ 1197us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:52.570218 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:52.572968 (+ 2750us) server_negotiation.cc:407] Connection header received
0811 01:47:52.575656 (+ 2688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:52.575687 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:52.576209 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:52.576700 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:52.578471 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:52.580400 (+ 1929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:52.581981 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:52.583087 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:52.583306 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:52.587655 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:52.587669 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:52.587698 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:52.587763 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:52.591906 (+ 4143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:52.592808 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:52.592817 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:52.592826 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:52.593063 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:52.593583 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:52.593589 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:52.593593 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:52.594430 (+ 837us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:52.594797 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:52.595922 (+ 1125us) server_negotiation.cc:299] Negotiation successful
0811 01:47:52.596538 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":884,"thread_start_us":549,"threads_started":1}
I20260811 01:47:52.598290 5933 leader_election.cc:302] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cc51b4cb4844ed8b52bb5752edd8770; no voters:
I20260811 01:47:52.599339 5933 leader_election.cc:288] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:47:52.599578 5939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:47:52.606613 5939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 1 LEADER]: Becoming Leader. State: Replica: 4cc51b4cb4844ed8b52bb5752edd8770, State: Running, Role: LEADER
I20260811 01:47:52.607409 5933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:47:52.607242 5939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36311 } }
I20260811 01:47:52.619261 5942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cc51b4cb4844ed8b52bb5752edd8770. Latest consensus state: current_term: 1 leader_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36311 } } }
I20260811 01:47:52.619735 5942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [sys.catalog]: This master's current role is: LEADER
I20260811 01:47:52.621466 5941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36311 } } }
I20260811 01:47:52.621800 5941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [sys.catalog]: This master's current role is: LEADER
W20260811 01:47:52.627010 5958 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 01:47:52.627184 5958 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 01:47:52.627317 5959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:47:52.629391 5959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:47:52.640128 5959 catalog_manager.cc:1347] Generated new cluster ID: 7b3df393207b46f4ab35f13706c35e33
I20260811 01:47:52.640316 5959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:47:52.674470 5959 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:47:52.675943 5959 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:47:52.684638 5959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770: Generated new TSK 0
I20260811 01:47:52.685711 5959 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
I20260811 01:47:52.764421 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:36311
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:47:53.135223 5965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:53.135573 5965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:53.135677 5965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:53.143392 5965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:53.143518 5965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:53.143635 5965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:47:53.153901 5965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:36311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:53.155277 5965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:53.156658 5965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:53.170433 5973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:53.170511 5970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:53.170507 5971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:53.171437 5965 server_base.cc:1034] running on GCE node
I20260811 01:47:53.172148 5965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:53.173250 5965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:53.174507 5965 hybrid_clock.cc:648] HybridClock initialized: now 1786412873174445 us; error 57 us; skew 500 ppm
I20260811 01:47:53.174877 5965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:53 dist-test-slave-9gcw krb5kdc[5842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412873, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:53.178359 5965 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:47:53.180051 5965 webserver.cc:460] Webserver started at http://127.21.214.65:42509/ using document root <none> and password file <none>
I20260811 01:47:53.180783 5965 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:53.180892 5965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:53.181288 5965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:47:53.185086 5965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "f8e3994507cd47bfa70535b694e81138"
format_stamp: "Formatted at 2026-08-11 01:47:53 on dist-test-slave-9gcw"
server_key: "6aabc93bb9a434f5367fc3ac453b40d3"
server_key_iv: "0561acd35f4832eb3ea678cd25c69e04"
server_key_version: "encryptionkey@0"
I20260811 01:47:53.186080 5965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "f8e3994507cd47bfa70535b694e81138"
format_stamp: "Formatted at 2026-08-11 01:47:53 on dist-test-slave-9gcw"
server_key: "6aabc93bb9a434f5367fc3ac453b40d3"
server_key_iv: "0561acd35f4832eb3ea678cd25c69e04"
server_key_version: "encryptionkey@0"
I20260811 01:47:53.192158 5965 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 01:47:53.196599 5981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:53.198221 5965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:47:53.198433 5965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "f8e3994507cd47bfa70535b694e81138"
format_stamp: "Formatted at 2026-08-11 01:47:53 on dist-test-slave-9gcw"
server_key: "6aabc93bb9a434f5367fc3ac453b40d3"
server_key_iv: "0561acd35f4832eb3ea678cd25c69e04"
server_key_version: "encryptionkey@0"
I20260811 01:47:53.198637 5965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:53.212575 5965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:53.217283 5965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:53.217615 5965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:53.218645 5965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:47:53.220073 5965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:47:53.220147 5965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:53.220279 5965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:47:53.220347 5965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:53.255533 5965 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:36407
I20260811 01:47:53.255577 6105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:36407 every 8 connection(s)
I20260811 01:47:53.256911 5965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:47:53.265677 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 5965
I20260811 01:47:53.265921 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:47:53.266919 22361 external_mini_cluster.cc:1428] Setting key 4081e311938e1edf1c55e9866f116af9
Aug 11 01:47:53 dist-test-slave-9gcw krb5kdc[5842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412873, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:47:53.279943 6108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:53.259711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:45459 (local address 127.21.214.126:36311)
0811 01:47:53.260206 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:53.260213 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:53.261300 (+ 1087us) server_negotiation.cc:407] Connection header received
0811 01:47:53.263190 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:53.263195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:53.263269 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:53.263387 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:53.265017 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:53.265902 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:53.268347 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:53.268979 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:53.269128 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:53.273822 (+ 4694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:53.273828 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:53.273832 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:53.273866 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:53.276558 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:53.277454 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:53.277459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:53.277463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:53.277596 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:53.278083 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:53.278090 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:53.278098 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:53.278382 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:53.278548 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:53.279310 (+ 762us) server_negotiation.cc:299] Negotiation successful
0811 01:47:53.279556 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":234,"threads_started":1}
I20260811 01:47:53.280987 6109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:53.260290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:45459)
0811 01:47:53.261065 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:53.261112 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:53.262906 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:53.263602 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:53.263610 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:53.264204 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:53.264828 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:53.264848 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:53.266124 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:53.266130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:53.268199 (+ 2069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:53.268212 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:53.269362 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:53.269372 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:53.269722 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:53.270543 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:53.270580 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:53.273546 (+ 2966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:53.276728 (+ 3182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:53.276739 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:53.276767 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:53.277322 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:53.277719 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:53.277724 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:53.277728 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:53.277947 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:53.278520 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:53.278528 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:53.278907 (+ 379us) client_negotiation.cc:769] Sending connection context
0811 01:47:53.279190 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 01:47:53.279802 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":270,"threads_started":1}
I20260811 01:47:53.282745 6106 heartbeater.cc:347] Connected to a master server at 127.21.214.126:36311
I20260811 01:47:53.283104 6106 heartbeater.cc:464] Registering TS with master...
I20260811 01:47:53.284060 6106 heartbeater.cc:511] Master 127.21.214.126:36311 requested a full tablet report, sending...
I20260811 01:47:53.286477 5894 ts_manager.cc:194] Registered new tserver with Master: f8e3994507cd47bfa70535b694e81138 (127.21.214.65:36407)
I20260811 01:47:53.288064 5894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:45459
WARNING: no policy specified for kudu/127.21.214.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
I20260811 01:47:53.337687 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:36311
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:47:53.702400 6114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:53.702803 6114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:53.702968 6114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:53.710912 6114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:53.711027 6114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:53.711241 6114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:47:53.721119 6114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:36311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:53.722502 6114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:53.723963 6114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:53.738598 6122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:53.738582 6120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:53.738917 6119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:53.739393 6114 server_base.cc:1034] running on GCE node
I20260811 01:47:53.740084 6114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:53.741199 6114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:53.742447 6114 hybrid_clock.cc:648] HybridClock initialized: now 1786412873742371 us; error 66 us; skew 500 ppm
I20260811 01:47:53.742889 6114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:53 dist-test-slave-9gcw krb5kdc[5842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412873, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:53.746711 6114 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:47:53.748304 6114 webserver.cc:460] Webserver started at http://127.21.214.66:41173/ using document root <none> and password file <none>
I20260811 01:47:53.749023 6114 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:53.749199 6114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:53.749524 6114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:47:53.752504 6114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "d99554359d414dddbf1fe58a2c00a555"
format_stamp: "Formatted at 2026-08-11 01:47:53 on dist-test-slave-9gcw"
server_key: "62fa4db0ab39d45910c0c6732cbe574d"
server_key_iv: "6ae26b2020fdbe122c69674de6d7d900"
server_key_version: "encryptionkey@0"
I20260811 01:47:53.753365 6114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "d99554359d414dddbf1fe58a2c00a555"
format_stamp: "Formatted at 2026-08-11 01:47:53 on dist-test-slave-9gcw"
server_key: "62fa4db0ab39d45910c0c6732cbe574d"
server_key_iv: "6ae26b2020fdbe122c69674de6d7d900"
server_key_version: "encryptionkey@0"
I20260811 01:47:53.759003 6114 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 01:47:53.762686 6130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:53.764161 6114 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:47:53.764379 6114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "d99554359d414dddbf1fe58a2c00a555"
format_stamp: "Formatted at 2026-08-11 01:47:53 on dist-test-slave-9gcw"
server_key: "62fa4db0ab39d45910c0c6732cbe574d"
server_key_iv: "6ae26b2020fdbe122c69674de6d7d900"
server_key_version: "encryptionkey@0"
I20260811 01:47:53.764569 6114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:53.781699 6114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:53.786211 6114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:53.786535 6114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:53.787580 6114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:47:53.789036 6114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:47:53.789144 6114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:53.789319 6114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:47:53.789402 6114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:53.826467 6114 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:44947
I20260811 01:47:53.826524 6263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:44947 every 8 connection(s)
I20260811 01:47:53.827816 6114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:47:53.828511 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 6114
I20260811 01:47:53.828718 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:47:53.829384 22361 external_mini_cluster.cc:1428] Setting key 48d0679a8113fe733aeaec5906947d67
Aug 11 01:47:53 dist-test-slave-9gcw krb5kdc[5842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412873, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:47:53.851547 6266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:53.830977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:47823 (local address 127.21.214.126:36311)
0811 01:47:53.831491 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:53.831498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:53.832717 (+ 1219us) server_negotiation.cc:407] Connection header received
0811 01:47:53.834634 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:53.834643 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:53.834751 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:53.834894 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:53.836469 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:53.837796 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:53.839466 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:53.840288 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:53.840459 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:53.844738 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:53.844745 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:53.844748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:53.844786 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:53.847846 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:53.848729 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:53.848743 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:53.848746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:53.848890 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:53.849479 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:53.849486 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:53.849490 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:53.849819 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:53.850020 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:53.850839 (+ 819us) server_negotiation.cc:299] Negotiation successful
0811 01:47:53.851116 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":270,"threads_started":1}
I20260811 01:47:53.852687 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:53.831587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:47823)
0811 01:47:53.832477 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:53.832531 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:53.834322 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:53.835116 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:53.835124 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:53.835658 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:53.836245 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:53.836264 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:53.837969 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:53.837972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:53.839298 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:53.839311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:53.840578 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:53.840584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:53.840900 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:53.841805 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:53.841828 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:53.844434 (+ 2606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:53.848031 (+ 3597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:53.848042 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:53.848062 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:53.848565 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:53.849022 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:53.849028 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:53.849032 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:53.849347 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:53.849977 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:53.849987 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:53.850415 (+ 428us) client_negotiation.cc:769] Sending connection context
0811 01:47:53.850737 (+ 322us) client_negotiation.cc:240] Negotiation successful
0811 01:47:53.851331 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":364,"threads_started":1}
I20260811 01:47:53.854655 6264 heartbeater.cc:347] Connected to a master server at 127.21.214.126:36311
I20260811 01:47:53.855047 6264 heartbeater.cc:464] Registering TS with master...
I20260811 01:47:53.856160 6264 heartbeater.cc:511] Master 127.21.214.126:36311 requested a full tablet report, sending...
I20260811 01:47:53.858170 5894 ts_manager.cc:194] Registered new tserver with Master: d99554359d414dddbf1fe58a2c00a555 (127.21.214.66:44947)
WARNING: no policy specified for kudu/127.21.214.67@KRBTEST.COM; defaulting to no policy
I20260811 01:47:53.859480 5894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:47823
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
I20260811 01:47:53.897671 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:36311
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:47:54.263453 6270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:54.263744 6270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:54.263849 6270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:54.271569 6270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:54.271704 6270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:54.271832 6270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:47:54.282387 6270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:36311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:54.283856 6270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:54.285291 6270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 01:47:54.292322 6106 heartbeater.cc:503] Master 127.21.214.126:36311 was elected leader, sending a full tablet report...
W20260811 01:47:54.300801 6276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:54.300879 6277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:54.300827 6279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:54.301741 6270 server_base.cc:1034] running on GCE node
I20260811 01:47:54.302501 6270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:54.303606 6270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:54.304847 6270 hybrid_clock.cc:648] HybridClock initialized: now 1786412874304790 us; error 54 us; skew 500 ppm
I20260811 01:47:54.305209 6270 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:54 dist-test-slave-9gcw krb5kdc[5842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412874, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:54.308657 6270 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:47:54.310302 6270 webserver.cc:460] Webserver started at http://127.21.214.67:33789/ using document root <none> and password file <none>
I20260811 01:47:54.311203 6270 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:54.311357 6270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:54.311687 6270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:47:54.315092 6270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "504a5cb11b0845a5a87ec23b38c6d522"
format_stamp: "Formatted at 2026-08-11 01:47:54 on dist-test-slave-9gcw"
server_key: "705e706b8bea8f0e6550536b71fbfb0b"
server_key_iv: "24249dc36759d3be667e1f20ecffe0b4"
server_key_version: "encryptionkey@0"
I20260811 01:47:54.316005 6270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "504a5cb11b0845a5a87ec23b38c6d522"
format_stamp: "Formatted at 2026-08-11 01:47:54 on dist-test-slave-9gcw"
server_key: "705e706b8bea8f0e6550536b71fbfb0b"
server_key_iv: "24249dc36759d3be667e1f20ecffe0b4"
server_key_version: "encryptionkey@0"
I20260811 01:47:54.321867 6270 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 01:47:54.325829 6286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:54.327561 6270 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:47:54.327725 6270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "504a5cb11b0845a5a87ec23b38c6d522"
format_stamp: "Formatted at 2026-08-11 01:47:54 on dist-test-slave-9gcw"
server_key: "705e706b8bea8f0e6550536b71fbfb0b"
server_key_iv: "24249dc36759d3be667e1f20ecffe0b4"
server_key_version: "encryptionkey@0"
I20260811 01:47:54.327970 6270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:54.344152 6270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:54.348750 6270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:54.349071 6270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:54.350126 6270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:47:54.351496 6270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:47:54.351572 6270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:54.351678 6270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:47:54.351716 6270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:54.387125 6270 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:38837
I20260811 01:47:54.387163 6415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:38837 every 8 connection(s)
I20260811 01:47:54.388522 6270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:47:54.398165 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 6270
I20260811 01:47:54.398388 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:47:54.398847 22361 external_mini_cluster.cc:1428] Setting key 5a745a41a1c0a5244f7a79415bd1d121
Aug 11 01:47:54 dist-test-slave-9gcw krb5kdc[5842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412874, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:47:54.410831 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:54.391507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:39483 (local address 127.21.214.126:36311)
0811 01:47:54.392053 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:54.392058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:54.392935 (+ 877us) server_negotiation.cc:407] Connection header received
0811 01:47:54.394900 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:54.394906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:54.395006 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:54.395131 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:54.396742 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.397755 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.399551 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.400103 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.400241 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:54.404372 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:54.404377 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:54.404380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:54.404413 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:54.407387 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:54.408120 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:54.408124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:54.408126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:54.408278 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:54.408784 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:54.408788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:54.408789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:54.409074 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:54.409278 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:54.410190 (+ 912us) server_negotiation.cc:299] Negotiation successful
0811 01:47:54.410426 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":290,"threads_started":1}
I20260811 01:47:54.411990 6419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:54.392019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:39483)
0811 01:47:54.392754 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:54.392802 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:54.394611 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:54.395411 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:54.395420 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:54.395948 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:54.396551 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:54.396568 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.397949 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.397953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:54.399394 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:54.399407 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.400366 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.400377 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:54.400625 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:54.401368 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:54.401392 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:54.404163 (+ 2771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:54.407531 (+ 3368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:54.407538 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:54.407552 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:54.408006 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:54.408434 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:54.408437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:54.408439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:54.408683 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:54.409251 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:54.409257 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:54.409677 (+ 420us) client_negotiation.cc:769] Sending connection context
0811 01:47:54.410086 (+ 409us) client_negotiation.cc:240] Negotiation successful
0811 01:47:54.410654 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":282,"threads_started":1}
I20260811 01:47:54.413704 6416 heartbeater.cc:347] Connected to a master server at 127.21.214.126:36311
I20260811 01:47:54.414113 6416 heartbeater.cc:464] Registering TS with master...
I20260811 01:47:54.415081 6416 heartbeater.cc:511] Master 127.21.214.126:36311 requested a full tablet report, sending...
I20260811 01:47:54.416584 5894 ts_manager.cc:194] Registered new tserver with Master: 504a5cb11b0845a5a87ec23b38c6d522 (127.21.214.67:38837)
I20260811 01:47:54.417519 5894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:39483
I20260811 01:47:54.427989 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:47:54.447059 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:54.433083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52454 (local address 127.21.214.126:36311)
0811 01:47:54.433326 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:54.433335 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:54.433934 (+ 599us) server_negotiation.cc:407] Connection header received
0811 01:47:54.434079 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:54.434083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:54.434152 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:54.434229 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:54.435860 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.436627 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.438284 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.438707 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.438813 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:54.441549 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:54.441554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:54.441557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:54.441592 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:54.444520 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:54.445273 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:54.445277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:54.445279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:54.445413 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:54.445790 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:54.445793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:54.445795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:54.446118 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:54.446303 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:54.446604 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 01:47:54.446819 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 01:47:54.451529 5894 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:47:54.458953 5894 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:47:54.486057 6436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:54.472824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41472 (local address 127.21.214.66:44947)
0811 01:47:54.474518 (+ 1694us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:54.474532 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:54.476123 (+ 1591us) server_negotiation.cc:407] Connection header received
0811 01:47:54.476618 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:54.476627 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:54.476925 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:54.477131 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:47:54.478277 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.479852 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.482795 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.484403 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.484546 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:54.484775 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:54.485483 (+ 708us) server_negotiation.cc:299] Negotiation successful
0811 01:47:54.485671 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1468,"thread_start_us":1271,"threads_started":1}
I20260811 01:47:54.486246 6432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:54.472002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:36407 (local address 127.0.0.1:37628)
0811 01:47:54.476669 (+ 4667us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:54.476692 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:54.476852 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:54.478228 (+ 1376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:54.478235 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:54.478268 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:47:54.478713 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:54.478728 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.480838 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.480844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:54.483074 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:54.483086 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.485261 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.485268 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:54.485435 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:54.485472 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 01:47:54.485603 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 01:47:54.485820 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2899,"mutex_wait_us":28,"thread_start_us":2707,"threads_started":1}
I20260811 01:47:54.486464 6437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:54.476131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:38837 (local address 127.0.0.1:40798)
0811 01:47:54.478473 (+ 2342us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:54.478497 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:54.478653 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:54.479542 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:54.479546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:54.479576 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:47:54.479994 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:54.480005 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.481632 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.481639 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:54.483458 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:54.483473 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.485584 (+ 2111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.485591 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:54.485748 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:54.485782 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 01:47:54.485876 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 01:47:54.486010 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2192,"mutex_wait_us":56,"thread_start_us":1998,"threads_started":1}
I20260811 01:47:54.486779 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:54.472232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37628 (local address 127.21.214.65:36407)
0811 01:47:54.477555 (+ 5323us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:54.477566 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:54.477596 (+ 30us) server_negotiation.cc:407] Connection header received
0811 01:47:54.477712 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:54.477719 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:54.477972 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:54.478855 (+ 883us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:47:54.479064 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.480683 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.483239 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.484997 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.485197 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:54.485443 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:54.485656 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 01:47:54.485820 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5138,"thread_start_us":4985,"threads_started":1}
I20260811 01:47:54.486882 6435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:54.472213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:44947 (local address 127.0.0.1:41472)
0811 01:47:54.475973 (+ 3760us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:54.476016 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:54.476325 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:54.477518 (+ 1193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:54.477529 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:54.477569 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:47:54.477995 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:54.478015 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.480027 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.480038 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:54.482599 (+ 2561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:54.482617 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.484905 (+ 2288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.484912 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:54.485130 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:54.485224 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 01:47:54.486362 (+ 1138us) client_negotiation.cc:240] Negotiation successful
0811 01:47:54.486528 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3611,"thread_start_us":3428,"threads_started":1}
I20260811 01:47:54.487383 6438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:54.476696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40798 (local address 127.21.214.67:38837)
0811 01:47:54.477947 (+ 1251us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:54.477954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:54.478629 (+ 675us) server_negotiation.cc:407] Connection header received
0811 01:47:54.478823 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:54.478830 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:54.479024 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:54.479206 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:47:54.480220 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.481487 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.483648 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.485039 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.485185 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:54.485425 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:54.486890 (+ 1465us) server_negotiation.cc:299] Negotiation successful
0811 01:47:54.487085 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1073,"thread_start_us":918,"threads_started":1}
I20260811 01:47:54.490815 6033 tablet_service.cc:1467] Processing CreateTablet for tablet b2f9f74a35ae4956a68c1998f60de208 (DEFAULT_TABLE table=test-table [id=6f36523ba694460486e3bd22c1352c27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:47:54.490829 6341 tablet_service.cc:1467] Processing CreateTablet for tablet b2f9f74a35ae4956a68c1998f60de208 (DEFAULT_TABLE table=test-table [id=6f36523ba694460486e3bd22c1352c27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:47:54.490829 6187 tablet_service.cc:1467] Processing CreateTablet for tablet b2f9f74a35ae4956a68c1998f60de208 (DEFAULT_TABLE table=test-table [id=6f36523ba694460486e3bd22c1352c27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:47:54.492256 6033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2f9f74a35ae4956a68c1998f60de208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:47:54.492311 6341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2f9f74a35ae4956a68c1998f60de208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:47:54.492311 6187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2f9f74a35ae4956a68c1998f60de208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:47:54.505165 6439 tablet_bootstrap.cc:492] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138: Bootstrap starting.
I20260811 01:47:54.507957 6440 tablet_bootstrap.cc:492] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555: Bootstrap starting.
I20260811 01:47:54.509317 6439 tablet_bootstrap.cc:654] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138: Neither blocks nor log segments found. Creating new log.
I20260811 01:47:54.510739 6439 log.cc:824] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138: Log is configured to *not* fsync() on all Append() calls
I20260811 01:47:54.512046 6440 tablet_bootstrap.cc:654] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555: Neither blocks nor log segments found. Creating new log.
I20260811 01:47:54.513375 6440 log.cc:824] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555: Log is configured to *not* fsync() on all Append() calls
I20260811 01:47:54.513998 6441 tablet_bootstrap.cc:492] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522: Bootstrap starting.
I20260811 01:47:54.514882 6439 tablet_bootstrap.cc:492] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138: No bootstrap required, opened a new log
I20260811 01:47:54.515259 6439 ts_tablet_manager.cc:1397] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20260811 01:47:54.517616 6440 tablet_bootstrap.cc:492] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555: No bootstrap required, opened a new log
I20260811 01:47:54.517935 6440 ts_tablet_manager.cc:1397] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260811 01:47:54.517980 6441 tablet_bootstrap.cc:654] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522: Neither blocks nor log segments found. Creating new log.
I20260811 01:47:54.519485 6441 log.cc:824] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522: Log is configured to *not* fsync() on all Append() calls
I20260811 01:47:54.523023 6441 tablet_bootstrap.cc:492] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522: No bootstrap required, opened a new log
I20260811 01:47:54.523327 6441 ts_tablet_manager.cc:1397] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 01:47:54.523864 6439 raft_consensus.cc:348] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } } peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } }
I20260811 01:47:54.524341 6439 raft_consensus.cc:374] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:47:54.524447 6439 raft_consensus.cc:729] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8e3994507cd47bfa70535b694e81138, State: Initialized, Role: FOLLOWER
I20260811 01:47:54.525061 6439 consensus_queue.cc:260] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } } peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } }
I20260811 01:47:54.528028 6439 ts_tablet_manager.cc:1428] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138: Time spent starting tablet: real 0.013s user 0.007s sys 0.004s
I20260811 01:47:54.528440 6440 raft_consensus.cc:348] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } } peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } }
I20260811 01:47:54.528893 6440 raft_consensus.cc:374] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:47:54.528985 6440 raft_consensus.cc:729] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d99554359d414dddbf1fe58a2c00a555, State: Initialized, Role: FOLLOWER
I20260811 01:47:54.529614 6440 consensus_queue.cc:260] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } } peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } }
I20260811 01:47:54.530843 6264 heartbeater.cc:503] Master 127.21.214.126:36311 was elected leader, sending a full tablet report...
I20260811 01:47:54.531549 6440 ts_tablet_manager.cc:1428] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260811 01:47:54.534695 6441 raft_consensus.cc:348] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } } peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } }
I20260811 01:47:54.535091 6441 raft_consensus.cc:374] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:47:54.535210 6441 raft_consensus.cc:729] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 504a5cb11b0845a5a87ec23b38c6d522, State: Initialized, Role: FOLLOWER
I20260811 01:47:54.535876 6441 consensus_queue.cc:260] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } } peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } }
I20260811 01:47:54.537119 6416 heartbeater.cc:503] Master 127.21.214.126:36311 was elected leader, sending a full tablet report...
I20260811 01:47:54.537645 6441 ts_tablet_manager.cc:1428] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522: Time spent starting tablet: real 0.014s user 0.011s sys 0.005s
W20260811 01:47:54.580240 6265 tablet.cc:2369] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:47:54.603360 6447 raft_consensus.cc:482] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:47:54.603708 6447 raft_consensus.cc:504] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } } peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } }
I20260811 01:47:54.605844 6447 leader_election.cc:288] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8e3994507cd47bfa70535b694e81138 (127.21.214.65:36407), 504a5cb11b0845a5a87ec23b38c6d522 (127.21.214.67:38837)
I20260811 01:47:54.613755 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:54.606316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:49241 (local address 127.21.214.65:36407)
0811 01:47:54.606565 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:54.606574 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:54.607569 (+ 995us) server_negotiation.cc:407] Connection header received
0811 01:47:54.607766 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:54.607771 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:54.607859 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:54.607965 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:47:54.608743 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.609766 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.611805 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.612649 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.612849 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:54.612972 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:54.613333 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 01:47:54.613468 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 01:47:54.613852 6438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:54.606459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:55059 (local address 127.21.214.67:38837)
0811 01:47:54.606688 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:54.606697 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:54.607765 (+ 1068us) server_negotiation.cc:407] Connection header received
0811 01:47:54.608046 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:54.608051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:54.608138 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:54.608265 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:47:54.608989 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.609807 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.611929 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.612723 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.612942 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:54.613029 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:54.613426 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 01:47:54.613559 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 01:47:54.613940 6449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:54.606300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:36407 (local address 127.21.214.66:49241)
0811 01:47:54.607440 (+ 1140us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:54.607457 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:54.607595 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:54.608000 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:54.608005 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:54.608031 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:47:54.608557 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:54.608570 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.609910 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.609917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:54.611630 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:54.611643 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.612993 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.613003 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:54.613134 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:54.613185 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 01:47:54.613300 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 01:47:54.613456 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":984,"mutex_wait_us":2,"thread_start_us":733,"threads_started":1}
I20260811 01:47:54.614573 6450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:54.606370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:38837 (local address 127.21.214.66:55059)
0811 01:47:54.607614 (+ 1244us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:54.607669 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:54.607870 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:54.608328 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:54.608332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:54.608354 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:47:54.608823 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:54.608836 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.609975 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.609979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:54.611763 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:54.611776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.613029 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.613036 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:54.613262 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:54.613287 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 01:47:54.613394 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 01:47:54.613544 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1088,"mutex_wait_us":3,"thread_start_us":823,"threads_started":1}
I20260811 01:47:54.615128 6056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f9f74a35ae4956a68c1998f60de208" candidate_uuid: "d99554359d414dddbf1fe58a2c00a555" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8e3994507cd47bfa70535b694e81138" is_pre_election: true
I20260811 01:47:54.615654 6366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f9f74a35ae4956a68c1998f60de208" candidate_uuid: "d99554359d414dddbf1fe58a2c00a555" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "504a5cb11b0845a5a87ec23b38c6d522" is_pre_election: true
I20260811 01:47:54.615880 6056 raft_consensus.cc:2454] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d99554359d414dddbf1fe58a2c00a555 in term 0.
I20260811 01:47:54.616266 6366 raft_consensus.cc:2454] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d99554359d414dddbf1fe58a2c00a555 in term 0.
I20260811 01:47:54.616729 6131 leader_election.cc:302] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d99554359d414dddbf1fe58a2c00a555, f8e3994507cd47bfa70535b694e81138; no voters:
I20260811 01:47:54.617375 6447 raft_consensus.cc:2781] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:47:54.617532 6447 raft_consensus.cc:482] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:47:54.617658 6447 raft_consensus.cc:3037] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:47:54.619994 6447 raft_consensus.cc:504] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } } peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } }
I20260811 01:47:54.620658 6447 leader_election.cc:288] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [CANDIDATE]: Term 1 election: Requested vote from peers f8e3994507cd47bfa70535b694e81138 (127.21.214.65:36407), 504a5cb11b0845a5a87ec23b38c6d522 (127.21.214.67:38837)
I20260811 01:47:54.621206 6056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f9f74a35ae4956a68c1998f60de208" candidate_uuid: "d99554359d414dddbf1fe58a2c00a555" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8e3994507cd47bfa70535b694e81138"
I20260811 01:47:54.621271 6366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f9f74a35ae4956a68c1998f60de208" candidate_uuid: "d99554359d414dddbf1fe58a2c00a555" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "504a5cb11b0845a5a87ec23b38c6d522"
I20260811 01:47:54.621441 6056 raft_consensus.cc:3037] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:47:54.621505 6366 raft_consensus.cc:3037] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:47:54.623788 6056 raft_consensus.cc:2454] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d99554359d414dddbf1fe58a2c00a555 in term 1.
I20260811 01:47:54.624320 6131 leader_election.cc:302] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d99554359d414dddbf1fe58a2c00a555, f8e3994507cd47bfa70535b694e81138; no voters:
I20260811 01:47:54.624364 6366 raft_consensus.cc:2454] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d99554359d414dddbf1fe58a2c00a555 in term 1.
I20260811 01:47:54.624629 6447 raft_consensus.cc:2781] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:47:54.625326 6447 raft_consensus.cc:686] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [term 1 LEADER]: Becoming Leader. State: Replica: d99554359d414dddbf1fe58a2c00a555, State: Running, Role: LEADER
I20260811 01:47:54.625936 6447 consensus_queue.cc:237] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } } peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } }
I20260811 01:47:54.630918 5894 catalog_manager.cc:5495] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 reported cstate change: term changed from 0 to 1, leader changed from <none> to d99554359d414dddbf1fe58a2c00a555 (127.21.214.66). New cstate: current_term: 1 leader_uuid: "d99554359d414dddbf1fe58a2c00a555" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } health_report { overall_health: UNKNOWN } } }
W20260811 01:47:54.640321 6417 tablet.cc:2369] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:47:54.680583 6436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:54.674493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41478 (local address 127.21.214.66:44947)
0811 01:47:54.674766 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:54.674771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:54.674791 (+ 20us) server_negotiation.cc:407] Connection header received
0811 01:47:54.674909 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:54.674912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:54.674992 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:54.675119 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:47:54.675559 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.676403 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.678207 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.678643 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.678779 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:54.679310 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:47:54.679891 (+ 581us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:47:54.680014 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:54.680168 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 01:47:54.680244 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 01:47:54.690119 6366 raft_consensus.cc:1261] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522 [term 1 FOLLOWER]: Refusing update from remote peer d99554359d414dddbf1fe58a2c00a555: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:47:54.690120 6056 raft_consensus.cc:1261] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138 [term 1 FOLLOWER]: Refusing update from remote peer d99554359d414dddbf1fe58a2c00a555: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:47:54.691399 6451 consensus_queue.cc:1035] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [LEADER]: Connected to new peer: Peer: permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:47:54.691881 6447 consensus_queue.cc:1035] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:47:54.705338 6455 mvcc.cc:205] Tried to move back new op lower bound from 7317147134717956096 to 7317147134471979008. Current Snapshot: MvccSnapshot[applied={T|T < 7317147134717956096}]
I20260811 01:47:54.720125 5893 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:47:54.720479 5893 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:47:54.724498 5893 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770: Sending DeleteTablet for 3 replicas of tablet b2f9f74a35ae4956a68c1998f60de208
I20260811 01:47:54.725796 6033 tablet_service.cc:1514] Processing DeleteTablet for tablet b2f9f74a35ae4956a68c1998f60de208 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:47:54 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:37628
I20260811 01:47:54.725983 6341 tablet_service.cc:1514] Processing DeleteTablet for tablet b2f9f74a35ae4956a68c1998f60de208 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:47:54 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:40798
I20260811 01:47:54.726222 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 5965
I20260811 01:47:54.726310 6187 tablet_service.cc:1514] Processing DeleteTablet for tablet b2f9f74a35ae4956a68c1998f60de208 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:47:54 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:41472
I20260811 01:47:54.737043 6465 tablet_replica.cc:330] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138: stopping tablet replica
I20260811 01:47:54.737879 6465 raft_consensus.cc:2229] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:47:54.738660 6465 raft_consensus.cc:2258] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:47:54.741676 6465 ts_tablet_manager.cc:1905] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:47:54.742388 6466 tablet_replica.cc:330] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522: stopping tablet replica
I20260811 01:47:54.747277 6466 raft_consensus.cc:2229] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:47:54.747870 6466 raft_consensus.cc:2258] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:47:54.750669 6465 ts_tablet_manager.cc:1918] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:47:54.750911 6465 log.cc:1196] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/b2f9f74a35ae4956a68c1998f60de208
I20260811 01:47:54.751652 6465 ts_tablet_manager.cc:1939] T b2f9f74a35ae4956a68c1998f60de208 P f8e3994507cd47bfa70535b694e81138: Deleting consensus metadata
Aug 11 01:47:54 dist-test-slave-9gcw krb5kdc[5842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:47:54.754027 5875 catalog_manager.cc:4841] TS f8e3994507cd47bfa70535b694e81138 (127.21.214.65:36407): tablet b2f9f74a35ae4956a68c1998f60de208 (table test-table [id=6f36523ba694460486e3bd22c1352c27]) successfully deleted
I20260811 01:47:54.754655 6466 ts_tablet_manager.cc:1905] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:47:54.755098 6467 tablet_replica.cc:330] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555: stopping tablet replica
I20260811 01:47:54.755604 6467 raft_consensus.cc:2229] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:47:54.756348 6467 raft_consensus.cc:2258] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:47:54.758708 6467 ts_tablet_manager.cc:1905] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:47:54.762759 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:54.727644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37630 (local address 127.21.214.65:36407)
0811 01:47:54.727886 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:54.727896 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:54.738411 (+ 10515us) server_negotiation.cc:407] Connection header received
0811 01:47:54.738976 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:54.738982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:54.739132 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:54.739218 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:54.741967 (+ 2749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.743059 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.745212 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.745874 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.745966 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:54.753837 (+ 7871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:54.753855 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:54.753873 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:54.753919 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:54.758148 (+ 4229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:54.759197 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:54.759206 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:54.759209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:54.759482 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:54.760082 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:54.760087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:54.760090 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:54.760716 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:54.761024 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:54.762117 (+ 1093us) server_negotiation.cc:299] Negotiation successful
0811 01:47:54.762366 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 01:47:54.763271 6466 ts_tablet_manager.cc:1918] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:47:54.763454 6466 log.cc:1196] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/b2f9f74a35ae4956a68c1998f60de208
I20260811 01:47:54.764036 6466 ts_tablet_manager.cc:1939] T b2f9f74a35ae4956a68c1998f60de208 P 504a5cb11b0845a5a87ec23b38c6d522: Deleting consensus metadata
I20260811 01:47:54.764374 6101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:54.766242 5877 catalog_manager.cc:4841] TS 504a5cb11b0845a5a87ec23b38c6d522 (127.21.214.67:38837): tablet b2f9f74a35ae4956a68c1998f60de208 (table test-table [id=6f36523ba694460486e3bd22c1352c27]) successfully deleted
I20260811 01:47:54.769110 6467 ts_tablet_manager.cc:1918] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:47:54.769520 6467 log.cc:1196] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/b2f9f74a35ae4956a68c1998f60de208
I20260811 01:47:54.770313 6467 ts_tablet_manager.cc:1939] T b2f9f74a35ae4956a68c1998f60de208 P d99554359d414dddbf1fe58a2c00a555: Deleting consensus metadata
I20260811 01:47:54.772475 5877 catalog_manager.cc:4841] TS d99554359d414dddbf1fe58a2c00a555 (127.21.214.66:44947): tablet b2f9f74a35ae4956a68c1998f60de208 (table test-table [id=6f36523ba694460486e3bd22c1352c27]) successfully deleted
I20260811 01:47:54.873042 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 5965
I20260811 01:47:54.904188 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 6114
Aug 11 01:47:54 dist-test-slave-9gcw krb5kdc[5842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.66@KRBTEST.COM
I20260811 01:47:54.929430 6436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:54.905053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41494 (local address 127.21.214.66:44947)
0811 01:47:54.905281 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:54.905293 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:54.905318 (+ 25us) server_negotiation.cc:407] Connection header received
0811 01:47:54.905544 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:54.905549 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:54.905615 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:54.905748 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:54.907163 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.908122 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.909833 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:54.910357 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:54.910511 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:54.920623 (+ 10112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:54.920646 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:54.920670 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:54.920714 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:54.925207 (+ 4493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:54.926426 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:54.926438 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:54.926442 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:54.926668 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:54.927264 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:54.927270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:54.927273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:54.927897 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:54.928303 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:54.928596 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 01:47:54.928855 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 01:47:54.931135 6254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:55.057273 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 6114
I20260811 01:47:55.084625 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 6270
Aug 11 01:47:55 dist-test-slave-9gcw krb5kdc[5842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.67@KRBTEST.COM
I20260811 01:47:55.102497 6438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:55.085466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40810 (local address 127.21.214.67:38837)
0811 01:47:55.085647 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:55.085681 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:55.085748 (+ 67us) server_negotiation.cc:407] Connection header received
0811 01:47:55.085971 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:55.085976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:55.086049 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:55.086162 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:55.087622 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:55.088421 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:55.089993 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:55.090431 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:55.090553 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:55.096033 (+ 5480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:55.096053 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:55.096073 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:55.096119 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:55.099224 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:55.100082 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:55.100090 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:55.100093 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:55.100237 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:55.100648 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:55.100654 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:55.100657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:55.101060 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:55.101476 (+ 416us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:55.101886 (+ 410us) server_negotiation.cc:299] Negotiation successful
0811 01:47:55.102101 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 01:47:55.103163 6411 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:55.195422 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 6270
I20260811 01:47:55.218453 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 5856
I20260811 01:47:55.219684 5928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:55.336934 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 5856
I20260811 01:47:55.366509 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:36311
--webserver_interface=127.21.214.126
--webserver_port=38411
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:36311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:47:55.726754 6476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:55.727074 6476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:55.727270 6476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:55.735196 6476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:47:55.735322 6476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:55.735435 6476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:55.735522 6476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:47:55.735585 6476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:47:55.746130 6476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:36311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:36311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=38411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:55.747640 6476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:55.749014 6476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:55.775970 6483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:55.775954 6484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:55.776291 6486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:55.777022 6476 server_base.cc:1034] running on GCE node
I20260811 01:47:55.778012 6476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:55.779758 6476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:55.781049 6476 hybrid_clock.cc:648] HybridClock initialized: now 1786412875780969 us; error 62 us; skew 500 ppm
I20260811 01:47:55.781554 6476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:55 dist-test-slave-9gcw krb5kdc[5842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412875, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:55.790127 6476 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:47:55.792137 6476 webserver.cc:460] Webserver started at http://127.21.214.126:38411/ using document root <none> and password file <none>
I20260811 01:47:55.792991 6476 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:55.793241 6476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:55.800185 6476 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 01:47:55.803473 6493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:55.805294 6476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 01:47:55.805529 6476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "4cc51b4cb4844ed8b52bb5752edd8770"
format_stamp: "Formatted at 2026-08-11 01:47:52 on dist-test-slave-9gcw"
server_key: "8f1e18d709bba2fb3e650fb36e0b3daa"
server_key_iv: "eef40a02dec53de7af4fbce3d8f76c08"
server_key_version: "encryptionkey@0"
I20260811 01:47:55.806883 6476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:55.831487 6476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:55.836306 6476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:55.836586 6476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:55.857420 6476 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:36311
I20260811 01:47:55.857450 6551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:36311 every 8 connection(s)
I20260811 01:47:55.859215 6476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:47:55.866954 6552 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:47:55.867745 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 6476
I20260811 01:47:55.868827 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:36407
--local_ip_for_outbound_sockets=127.21.214.65
--tserver_master_addrs=127.21.214.126:36311
--webserver_port=42509
--webserver_interface=127.21.214.65
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:47:55.872646 6552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770: Bootstrap starting.
I20260811 01:47:55.896322 6552 log.cc:824] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770: Log is configured to *not* fsync() on all Append() calls
I20260811 01:47:55.909221 6552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:47:55.909811 6552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770: Bootstrap complete.
I20260811 01:47:55.919713 6552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36311 } }
I20260811 01:47:55.920568 6552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cc51b4cb4844ed8b52bb5752edd8770, State: Initialized, Role: FOLLOWER
I20260811 01:47:55.921368 6552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36311 } }
I20260811 01:47:55.921620 6552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:47:55.921742 6552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:47:55.921942 6552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:47:55.926033 6552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36311 } }
I20260811 01:47:55.926568 6552 leader_election.cc:302] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cc51b4cb4844ed8b52bb5752edd8770; no voters:
I20260811 01:47:55.927268 6552 leader_election.cc:288] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 01:47:55.927397 6557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 01:47:55.928767 6557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 2 LEADER]: Becoming Leader. State: Replica: 4cc51b4cb4844ed8b52bb5752edd8770, State: Running, Role: LEADER
I20260811 01:47:55.929370 6557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36311 } }
I20260811 01:47:55.929915 6552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:47:55.932250 6560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cc51b4cb4844ed8b52bb5752edd8770. Latest consensus state: current_term: 2 leader_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36311 } } }
I20260811 01:47:55.932677 6560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [sys.catalog]: This master's current role is: LEADER
I20260811 01:47:55.933354 6559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36311 } } }
I20260811 01:47:55.933658 6559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [sys.catalog]: This master's current role is: LEADER
I20260811 01:47:55.938776 6566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:47:55.949240 6566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:47:55.951190 6566 catalog_manager.cc:1259] Loaded cluster ID: 7b3df393207b46f4ab35f13706c35e33
I20260811 01:47:55.951332 6566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:47:55.955691 6566 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 01:47:55.980698 6566 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 01:47:56.002085 6566 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:47:56.004150 6566 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770: Loaded TSK: 0
I20260811 01:47:56.005904 6566 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 01:47:56.259003 6554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:56.259342 6554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:56.259513 6554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:56.268006 6554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:56.268226 6554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:56.268421 6554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:47:56.278815 6554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:36407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=42509
--tserver_master_addrs=127.21.214.126:36311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:56.280284 6554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:56.281859 6554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:56.296490 6585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:56.297000 6584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:56.298664 6588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:56.299433 6554 server_base.cc:1034] running on GCE node
I20260811 01:47:56.300129 6554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:56.301318 6554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:56.302572 6554 hybrid_clock.cc:648] HybridClock initialized: now 1786412876302487 us; error 61 us; skew 500 ppm
I20260811 01:47:56.302937 6554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:56 dist-test-slave-9gcw krb5kdc[5842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412876, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:56.307024 6554 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:47:56.309007 6554 webserver.cc:460] Webserver started at http://127.21.214.65:42509/ using document root <none> and password file <none>
I20260811 01:47:56.309901 6554 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:56.310097 6554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:56.316877 6554 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 01:47:56.320261 6595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:56.321997 6554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:47:56.322202 6554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "f8e3994507cd47bfa70535b694e81138"
format_stamp: "Formatted at 2026-08-11 01:47:53 on dist-test-slave-9gcw"
server_key: "6aabc93bb9a434f5367fc3ac453b40d3"
server_key_iv: "0561acd35f4832eb3ea678cd25c69e04"
server_key_version: "encryptionkey@0"
I20260811 01:47:56.323352 6554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:56.337603 6554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:56.342087 6554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:56.342381 6554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:56.343410 6554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:47:56.344914 6554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:47:56.345028 6554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:56.345202 6554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:47:56.345284 6554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:56.380275 6554 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:36407
I20260811 01:47:56.380403 6724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:36407 every 8 connection(s)
I20260811 01:47:56.381943 6554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:47:56.392649 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 6554
I20260811 01:47:56.393468 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:44947
--local_ip_for_outbound_sockets=127.21.214.66
--tserver_master_addrs=127.21.214.126:36311
--webserver_port=41173
--webserver_interface=127.21.214.66
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 01:47:56 dist-test-slave-9gcw krb5kdc[5842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412876, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:47:56.409547 6727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:56.385050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:54907 (local address 127.21.214.126:36311)
0811 01:47:56.385771 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:56.385783 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:56.386455 (+ 672us) server_negotiation.cc:407] Connection header received
0811 01:47:56.388632 (+ 2177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:56.388645 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:56.388980 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:56.389376 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:56.390871 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:56.392352 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:56.394644 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:56.395881 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:56.396105 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:56.400263 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:56.400277 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:56.400303 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:56.400358 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:56.404590 (+ 4232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:56.405733 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:56.405741 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:56.405749 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:56.405921 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:56.406468 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:56.406471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:56.406473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:56.406958 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:56.407105 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:56.408108 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0811 01:47:56.408450 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":374,"threads_started":1}
I20260811 01:47:56.410015 6728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:56.385341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:54907)
0811 01:47:56.386250 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:56.386313 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:56.388096 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:56.389523 (+ 1427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:56.389530 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:56.390059 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:56.390645 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:56.390662 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:56.392583 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:56.392588 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:56.394455 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:56.394471 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:56.396141 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:56.396146 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:56.396376 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:56.397281 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:56.397314 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:56.399898 (+ 2584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:56.404784 (+ 4886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:56.404797 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:56.404823 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:56.405574 (+ 751us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:56.406046 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:56.406054 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:56.406061 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:56.406335 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:56.407111 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:56.407123 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:56.407597 (+ 474us) client_negotiation.cc:769] Sending connection context
0811 01:47:56.407914 (+ 317us) client_negotiation.cc:240] Negotiation successful
0811 01:47:56.408556 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":408,"threads_started":1}
I20260811 01:47:56.413064 6725 heartbeater.cc:347] Connected to a master server at 127.21.214.126:36311
I20260811 01:47:56.413542 6725 heartbeater.cc:464] Registering TS with master...
I20260811 01:47:56.414327 6725 heartbeater.cc:511] Master 127.21.214.126:36311 requested a full tablet report, sending...
I20260811 01:47:56.416666 6513 ts_manager.cc:194] Registered new tserver with Master: f8e3994507cd47bfa70535b694e81138 (127.21.214.65:36407)
I20260811 01:47:56.418367 6513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:54907
W20260811 01:47:56.761031 6729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:56.761498 6729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:56.761685 6729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:56.769501 6729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:56.769723 6729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:56.769939 6729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:47:56.780180 6729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:44947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=41173
--tserver_master_addrs=127.21.214.126:36311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:56.781684 6729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:56.783140 6729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:56.797595 6737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:56.797551 6738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:56.797606 6741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:56.798817 6729 server_base.cc:1034] running on GCE node
I20260811 01:47:56.799517 6729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:56.800715 6729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:56.801959 6729 hybrid_clock.cc:648] HybridClock initialized: now 1786412876801901 us; error 57 us; skew 500 ppm
I20260811 01:47:56.802318 6729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:56 dist-test-slave-9gcw krb5kdc[5842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412876, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:56.805831 6729 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:47:56.807404 6729 webserver.cc:460] Webserver started at http://127.21.214.66:41173/ using document root <none> and password file <none>
I20260811 01:47:56.808153 6729 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:56.808327 6729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:56.814445 6729 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 01:47:56.817402 6750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:56.819052 6729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 01:47:56.819293 6729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "d99554359d414dddbf1fe58a2c00a555"
format_stamp: "Formatted at 2026-08-11 01:47:53 on dist-test-slave-9gcw"
server_key: "62fa4db0ab39d45910c0c6732cbe574d"
server_key_iv: "6ae26b2020fdbe122c69674de6d7d900"
server_key_version: "encryptionkey@0"
I20260811 01:47:56.820461 6729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:56.840409 6729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:56.845109 6729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:56.845481 6729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:56.846587 6729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:47:56.848065 6729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:47:56.848204 6729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:56.848352 6729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:47:56.848448 6729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:56.885543 6729 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:44947
I20260811 01:47:56.885634 6882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:44947 every 8 connection(s)
I20260811 01:47:56.887137 6729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:47:56.888466 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 6729
I20260811 01:47:56.889511 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:38837
--local_ip_for_outbound_sockets=127.21.214.67
--tserver_master_addrs=127.21.214.126:36311
--webserver_port=33789
--webserver_interface=127.21.214.67
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 01:47:56 dist-test-slave-9gcw krb5kdc[5842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412876, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:47:56.911695 6727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:56.890077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:39021 (local address 127.21.214.126:36311)
0811 01:47:56.893811 (+ 3734us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:56.893816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:56.893832 (+ 16us) server_negotiation.cc:407] Connection header received
0811 01:47:56.893886 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:56.893890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:56.893982 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:56.894108 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:56.895746 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:56.896457 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:56.898541 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:56.899337 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:56.899523 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:56.904576 (+ 5053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:56.904581 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:56.904585 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:56.904622 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:56.907507 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:56.908610 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:56.908614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:56.908616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:56.908817 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:56.909408 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:56.909412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:56.909414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:56.909763 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:56.909886 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:56.910971 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0811 01:47:56.911284 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3599}
I20260811 01:47:56.913136 6886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:56.890577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:39021)
0811 01:47:56.891476 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:56.891526 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:56.893372 (+ 1846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:56.894386 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:56.894395 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:56.894938 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:56.895542 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:56.895560 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:56.896682 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:56.896694 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:56.898339 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:56.898358 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:56.899641 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:56.899651 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:56.900042 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:56.900948 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:56.900972 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:56.904298 (+ 3326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:56.907687 (+ 3389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:56.907700 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:56.907733 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:56.908474 (+ 741us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:56.908943 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:56.908951 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:56.908957 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:56.909278 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:56.909911 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:56.909922 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:56.910456 (+ 534us) client_negotiation.cc:769] Sending connection context
0811 01:47:56.910827 (+ 371us) client_negotiation.cc:240] Negotiation successful
0811 01:47:56.911465 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":376,"threads_started":1}
I20260811 01:47:56.914964 6883 heartbeater.cc:347] Connected to a master server at 127.21.214.126:36311
I20260811 01:47:56.915362 6883 heartbeater.cc:464] Registering TS with master...
I20260811 01:47:56.916236 6883 heartbeater.cc:511] Master 127.21.214.126:36311 requested a full tablet report, sending...
I20260811 01:47:56.917976 6513 ts_manager.cc:194] Registered new tserver with Master: d99554359d414dddbf1fe58a2c00a555 (127.21.214.66:44947)
I20260811 01:47:56.918879 6513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:39021
W20260811 01:47:57.264322 6887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:57.264725 6887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:57.264925 6887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:57.273063 6887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:57.273284 6887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:57.273486 6887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:47:57.283727 6887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:38837
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=33789
--tserver_master_addrs=127.21.214.126:36311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:57.285238 6887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:57.286706 6887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:57.301800 6895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:57.301803 6893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:57.302146 6892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:57.302812 6887 server_base.cc:1034] running on GCE node
I20260811 01:47:57.303491 6887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:57.304459 6887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:57.305703 6887 hybrid_clock.cc:648] HybridClock initialized: now 1786412877305625 us; error 59 us; skew 500 ppm
I20260811 01:47:57.306181 6887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:57 dist-test-slave-9gcw krb5kdc[5842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412877, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:57.310154 6887 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:47:57.312125 6887 webserver.cc:460] Webserver started at http://127.21.214.67:33789/ using document root <none> and password file <none>
I20260811 01:47:57.312961 6887 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:57.313179 6887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:57.319921 6887 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 01:47:57.323436 6907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:57.325379 6887 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 01:47:57.325654 6887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "504a5cb11b0845a5a87ec23b38c6d522"
format_stamp: "Formatted at 2026-08-11 01:47:54 on dist-test-slave-9gcw"
server_key: "705e706b8bea8f0e6550536b71fbfb0b"
server_key_iv: "24249dc36759d3be667e1f20ecffe0b4"
server_key_version: "encryptionkey@0"
I20260811 01:47:57.326973 6887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:57.352330 6887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:57.357550 6887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:57.357884 6887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:57.359021 6887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:47:57.360558 6887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:47:57.360654 6887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:57.360841 6887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:47:57.360960 6887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:57.400663 6887 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:38837
I20260811 01:47:57.400719 7039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:38837 every 8 connection(s)
I20260811 01:47:57.402391 6887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:47:57.404094 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 6887
Aug 11 01:47:57 dist-test-slave-9gcw krb5kdc[5842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412877, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:47:57.421720 6725 heartbeater.cc:503] Master 127.21.214.126:36311 was elected leader, sending a full tablet report...
I20260811 01:47:57.424369 6727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.405389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52456 (local address 127.21.214.126:36311)
0811 01:47:57.405629 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:57.405636 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:57.406223 (+ 587us) server_negotiation.cc:407] Connection header received
0811 01:47:57.406554 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:57.406558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:57.406631 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:57.406765 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:57.408531 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.409401 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.411682 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.412618 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.412775 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:57.416860 (+ 4085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:57.416869 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:57.416878 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:57.416930 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:57.420624 (+ 3694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:57.421764 (+ 1140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:57.421774 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:57.421778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:57.421960 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:57.422475 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:57.422480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:57.422484 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:57.422919 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:57.423264 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:57.423692 (+ 428us) server_negotiation.cc:299] Negotiation successful
0811 01:47:57.423951 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":77}
I20260811 01:47:57.429600 7045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.405527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:35423 (local address 127.21.214.126:36311)
0811 01:47:57.406358 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:57.406363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:57.407325 (+ 962us) server_negotiation.cc:407] Connection header received
0811 01:47:57.409451 (+ 2126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:57.409458 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:57.409543 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:57.409636 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:57.411873 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.413292 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.416197 (+ 2905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.416819 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.416957 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:57.421727 (+ 4770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:57.421733 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:57.421781 (+ 48us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:57.421853 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:57.425794 (+ 3941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:57.426672 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:57.426680 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:57.426684 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:57.426827 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:57.427657 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:57.427662 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:57.427666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:57.428021 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:57.428181 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:57.429033 (+ 852us) server_negotiation.cc:299] Negotiation successful
0811 01:47:57.429291 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":739,"spinlock_wait_cycles":1792,"thread_start_us":643,"threads_started":1}
I20260811 01:47:57.430780 7046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.406153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:35423)
0811 01:47:57.407139 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:57.407196 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:57.409123 (+ 1927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:57.409988 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:57.410001 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:57.410801 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:47:57.411656 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:57.411688 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.413469 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.413473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:57.416001 (+ 2528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:57.416023 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.417069 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.417074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:57.417386 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:57.418233 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:47:57.418264 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:47:57.421440 (+ 3176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:47:57.425954 (+ 4514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:57.425968 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:57.425991 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:57.426510 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:57.427330 (+ 820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:47:57.427334 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:47:57.427336 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:47:57.427542 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:47:57.428169 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:47:57.428176 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:47:57.428599 (+ 423us) client_negotiation.cc:769] Sending connection context
0811 01:47:57.428907 (+ 308us) client_negotiation.cc:240] Negotiation successful
0811 01:47:57.429464 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":401,"threads_started":1}
I20260811 01:47:57.432221 7041 heartbeater.cc:347] Connected to a master server at 127.21.214.126:36311
I20260811 01:47:57.432538 7041 heartbeater.cc:464] Registering TS with master...
I20260811 01:47:57.433465 7041 heartbeater.cc:511] Master 127.21.214.126:36311 requested a full tablet report, sending...
I20260811 01:47:57.434914 6513 ts_manager.cc:194] Registered new tserver with Master: 504a5cb11b0845a5a87ec23b38c6d522 (127.21.214.67:38837)
I20260811 01:47:57.435729 6513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:35423
I20260811 01:47:57.439607 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:47:57.446657 7045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.440593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52470 (local address 127.21.214.126:36311)
0811 01:47:57.440775 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:57.440780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:57.441358 (+ 578us) server_negotiation.cc:407] Connection header received
0811 01:47:57.441535 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:57.441538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:57.441665 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:57.441793 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:47:57.442209 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.442907 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.444458 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.444933 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.445026 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:57.445619 (+ 593us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:47:57.446123 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:47:57.446256 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:57.446385 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 01:47:57.446455 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 01:47:57.447587 6513 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:52470:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:47:57.455013 6513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:47:57.478016 7054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.468181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37636 (local address 127.21.214.65:36407)
0811 01:47:57.469360 (+ 1179us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:57.469373 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:57.469414 (+ 41us) server_negotiation.cc:407] Connection header received
0811 01:47:57.469546 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:57.469558 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:57.469868 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:57.470128 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:47:57.471579 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.473274 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.475705 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.476691 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.476882 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:57.477159 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:57.477489 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 01:47:57.477656 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":939,"thread_start_us":784,"threads_started":1}
I20260811 01:47:57.478070 7052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.467340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:36407 (local address 127.0.0.1:37636)
0811 01:47:57.468936 (+ 1596us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:57.469033 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:57.469308 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:57.470157 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:57.470167 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:57.470212 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:47:57.470754 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:57.470768 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.473407 (+ 2639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.473412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:57.475537 (+ 2125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:57.475553 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.476969 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.476975 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:57.477212 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:57.477279 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 01:47:57.477442 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 01:47:57.477634 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1271,"mutex_wait_us":42,"thread_start_us":822,"threads_started":1}
I20260811 01:47:57.480075 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.470622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:38837 (local address 127.0.0.1:40826)
0811 01:47:57.471349 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:57.471371 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:57.471486 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:57.472732 (+ 1246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:57.472737 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:57.472762 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:47:57.473116 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:57.473125 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.475119 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.475124 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:57.477420 (+ 2296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:57.477435 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.479457 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.479463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:57.479630 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:57.479653 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 01:47:57.479730 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 01:47:57.479823 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":744,"threads_started":1}
I20260811 01:47:57.480337 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.471130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40826 (local address 127.21.214.67:38837)
0811 01:47:57.472199 (+ 1069us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:57.472205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:57.472230 (+ 25us) server_negotiation.cc:407] Connection header received
0811 01:47:57.472300 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:57.472304 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:57.472526 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:57.472732 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:47:57.473341 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.474949 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.477670 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.479093 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.479250 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:57.479491 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:57.479816 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 01:47:57.479983 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":915,"thread_start_us":748,"threads_started":1}
I20260811 01:47:57.482648 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.467403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:44947 (local address 127.0.0.1:41496)
0811 01:47:57.470460 (+ 3057us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:57.470477 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:57.470614 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:57.471965 (+ 1351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:57.471969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:57.472000 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:47:57.472320 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:57.472332 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.474378 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.474384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:57.476899 (+ 2515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:57.476915 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.481812 (+ 4897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.481819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:57.481988 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:57.482021 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 01:47:57.482131 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 01:47:57.482275 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2895,"mutex_wait_us":65,"thread_start_us":2737,"threads_started":1}
I20260811 01:47:57.482671 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.467923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41496 (local address 127.21.214.66:44947)
0811 01:47:57.469938 (+ 2015us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:57.469950 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:57.470880 (+ 930us) server_negotiation.cc:407] Connection header received
0811 01:47:57.471078 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:57.471086 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:57.471348 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:57.471721 (+ 373us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:47:57.472526 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.474177 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.477117 (+ 2940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.481582 (+ 4465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.481752 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:57.481948 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:57.482201 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 01:47:57.482362 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1877,"thread_start_us":1742,"threads_started":1}
I20260811 01:47:57.482780 6652 tablet_service.cc:1467] Processing CreateTablet for tablet 865ebd2dc2e74c3fa7cf4e4e41d42690 (DEFAULT_TABLE table=test-table [id=83a1e39d2ca2483ba66d2e0d23c9759a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:47:57.484508 6652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 865ebd2dc2e74c3fa7cf4e4e41d42690. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:47:57.485023 6963 tablet_service.cc:1467] Processing CreateTablet for tablet 865ebd2dc2e74c3fa7cf4e4e41d42690 (DEFAULT_TABLE table=test-table [id=83a1e39d2ca2483ba66d2e0d23c9759a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:47:57.486181 6807 tablet_service.cc:1467] Processing CreateTablet for tablet 865ebd2dc2e74c3fa7cf4e4e41d42690 (DEFAULT_TABLE table=test-table [id=83a1e39d2ca2483ba66d2e0d23c9759a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:47:57.486557 6963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 865ebd2dc2e74c3fa7cf4e4e41d42690. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:47:57.487190 6807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 865ebd2dc2e74c3fa7cf4e4e41d42690. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:47:57.503494 7059 tablet_bootstrap.cc:492] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555: Bootstrap starting.
I20260811 01:47:57.505038 7060 tablet_bootstrap.cc:492] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138: Bootstrap starting.
I20260811 01:47:57.505107 7062 tablet_bootstrap.cc:492] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522: Bootstrap starting.
I20260811 01:47:57.508280 7060 tablet_bootstrap.cc:654] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138: Neither blocks nor log segments found. Creating new log.
I20260811 01:47:57.508354 7059 tablet_bootstrap.cc:654] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555: Neither blocks nor log segments found. Creating new log.
I20260811 01:47:57.508703 7062 tablet_bootstrap.cc:654] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522: Neither blocks nor log segments found. Creating new log.
I20260811 01:47:57.509704 7059 log.cc:824] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555: Log is configured to *not* fsync() on all Append() calls
I20260811 01:47:57.510028 7062 log.cc:824] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522: Log is configured to *not* fsync() on all Append() calls
I20260811 01:47:57.510205 7060 log.cc:824] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138: Log is configured to *not* fsync() on all Append() calls
I20260811 01:47:57.515072 7060 tablet_bootstrap.cc:492] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138: No bootstrap required, opened a new log
I20260811 01:47:57.515419 7060 ts_tablet_manager.cc:1397] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260811 01:47:57.515615 7062 tablet_bootstrap.cc:492] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522: No bootstrap required, opened a new log
I20260811 01:47:57.515605 7059 tablet_bootstrap.cc:492] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555: No bootstrap required, opened a new log
I20260811 01:47:57.515908 7062 ts_tablet_manager.cc:1397] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.008s
I20260811 01:47:57.515911 7059 ts_tablet_manager.cc:1397] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260811 01:47:57.523304 7060 raft_consensus.cc:348] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } } peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } }
I20260811 01:47:57.523720 7060 raft_consensus.cc:374] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:47:57.523833 7060 raft_consensus.cc:729] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8e3994507cd47bfa70535b694e81138, State: Initialized, Role: FOLLOWER
I20260811 01:47:57.524619 7060 consensus_queue.cc:260] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } } peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } }
I20260811 01:47:57.526647 7060 ts_tablet_manager.cc:1428] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260811 01:47:57.527587 7062 raft_consensus.cc:348] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } } peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } }
I20260811 01:47:57.527608 7059 raft_consensus.cc:348] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } } peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } }
I20260811 01:47:57.528084 7059 raft_consensus.cc:374] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:47:57.528090 7062 raft_consensus.cc:374] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:47:57.528208 7062 raft_consensus.cc:729] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 504a5cb11b0845a5a87ec23b38c6d522, State: Initialized, Role: FOLLOWER
I20260811 01:47:57.528211 7059 raft_consensus.cc:729] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d99554359d414dddbf1fe58a2c00a555, State: Initialized, Role: FOLLOWER
I20260811 01:47:57.528990 7062 consensus_queue.cc:260] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } } peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } }
I20260811 01:47:57.528990 7059 consensus_queue.cc:260] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } } peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } }
I20260811 01:47:57.531257 7041 heartbeater.cc:503] Master 127.21.214.126:36311 was elected leader, sending a full tablet report...
I20260811 01:47:57.531504 7062 ts_tablet_manager.cc:1428] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522: Time spent starting tablet: real 0.015s user 0.017s sys 0.000s
I20260811 01:47:57.532886 6883 heartbeater.cc:503] Master 127.21.214.126:36311 was elected leader, sending a full tablet report...
I20260811 01:47:57.533641 7059 ts_tablet_manager.cc:1428] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555: Time spent starting tablet: real 0.018s user 0.009s sys 0.010s
I20260811 01:47:57.605062 7067 raft_consensus.cc:482] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:47:57.605487 7067 raft_consensus.cc:504] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } } peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } }
I20260811 01:47:57.607337 7067 leader_election.cc:288] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d99554359d414dddbf1fe58a2c00a555 (127.21.214.66:44947), 504a5cb11b0845a5a87ec23b38c6d522 (127.21.214.67:38837)
I20260811 01:47:57.615001 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.607756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:42417 (local address 127.21.214.66:44947)
0811 01:47:57.607960 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:57.607965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:57.608631 (+ 666us) server_negotiation.cc:407] Connection header received
0811 01:47:57.608878 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:57.608886 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:57.608968 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:57.609081 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:47:57.609825 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.610863 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.613062 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.613953 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.614073 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:57.614137 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:57.614589 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 01:47:57.614730 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 01:47:57.615088 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.608851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:43169 (local address 127.21.214.67:38837)
0811 01:47:57.609064 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:57.609070 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:57.609470 (+ 400us) server_negotiation.cc:407] Connection header received
0811 01:47:57.609593 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:57.609601 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:57.609682 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:57.609787 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:47:57.610399 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.611404 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.613324 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.614179 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.614349 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:57.614421 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:57.614698 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 01:47:57.614852 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 01:47:57.615199 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.608647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:38837 (local address 127.21.214.65:43169)
0811 01:47:57.609343 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:57.609366 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:57.609484 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:57.609812 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:57.609816 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:57.609840 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:47:57.610206 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:57.610244 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.611532 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.611535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:57.613154 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:57.613187 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.614386 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.614390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:57.614501 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:57.614559 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 01:47:57.614654 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 01:47:57.614783 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":1,"thread_start_us":331,"threads_started":1}
I20260811 01:47:57.615640 7070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.607748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:44947 (local address 127.21.214.65:42417)
0811 01:47:57.608516 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:57.608537 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:47:57.608691 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:47:57.609194 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:47:57.609199 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:47:57.609228 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:47:57.609633 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:57.609647 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.611022 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.611028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:57.612899 (+ 1871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:47:57.612912 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.614197 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.614204 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:47:57.614407 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:57.614443 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 01:47:57.614533 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 01:47:57.614649 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"mutex_wait_us":7,"thread_start_us":393,"threads_started":1}
I20260811 01:47:57.616246 6986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "865ebd2dc2e74c3fa7cf4e4e41d42690" candidate_uuid: "f8e3994507cd47bfa70535b694e81138" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "504a5cb11b0845a5a87ec23b38c6d522" is_pre_election: true
I20260811 01:47:57.616631 6831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "865ebd2dc2e74c3fa7cf4e4e41d42690" candidate_uuid: "f8e3994507cd47bfa70535b694e81138" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d99554359d414dddbf1fe58a2c00a555" is_pre_election: true
I20260811 01:47:57.616789 6986 raft_consensus.cc:2454] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8e3994507cd47bfa70535b694e81138 in term 0.
I20260811 01:47:57.617103 6831 raft_consensus.cc:2454] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8e3994507cd47bfa70535b694e81138 in term 0.
I20260811 01:47:57.617522 6600 leader_election.cc:302] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 504a5cb11b0845a5a87ec23b38c6d522, f8e3994507cd47bfa70535b694e81138; no voters:
I20260811 01:47:57.617908 7067 raft_consensus.cc:2781] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:47:57.618060 7067 raft_consensus.cc:482] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:47:57.618192 7067 raft_consensus.cc:3037] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:47:57.620514 7067 raft_consensus.cc:504] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } } peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } }
I20260811 01:47:57.621119 7067 leader_election.cc:288] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [CANDIDATE]: Term 1 election: Requested vote from peers d99554359d414dddbf1fe58a2c00a555 (127.21.214.66:44947), 504a5cb11b0845a5a87ec23b38c6d522 (127.21.214.67:38837)
I20260811 01:47:57.621650 6831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "865ebd2dc2e74c3fa7cf4e4e41d42690" candidate_uuid: "f8e3994507cd47bfa70535b694e81138" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d99554359d414dddbf1fe58a2c00a555"
I20260811 01:47:57.621774 6986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "865ebd2dc2e74c3fa7cf4e4e41d42690" candidate_uuid: "f8e3994507cd47bfa70535b694e81138" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "504a5cb11b0845a5a87ec23b38c6d522"
I20260811 01:47:57.621876 6831 raft_consensus.cc:3037] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:47:57.622021 6986 raft_consensus.cc:3037] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:47:57.624313 6831 raft_consensus.cc:2454] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8e3994507cd47bfa70535b694e81138 in term 1.
I20260811 01:47:57.624343 6986 raft_consensus.cc:2454] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8e3994507cd47bfa70535b694e81138 in term 1.
I20260811 01:47:57.624781 6600 leader_election.cc:302] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 504a5cb11b0845a5a87ec23b38c6d522, f8e3994507cd47bfa70535b694e81138; no voters:
I20260811 01:47:57.625097 7067 raft_consensus.cc:2781] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:47:57.625711 7067 raft_consensus.cc:686] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [term 1 LEADER]: Becoming Leader. State: Replica: f8e3994507cd47bfa70535b694e81138, State: Running, Role: LEADER
I20260811 01:47:57.626315 7067 consensus_queue.cc:237] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } } peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } }
I20260811 01:47:57.631345 6513 catalog_manager.cc:5495] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8e3994507cd47bfa70535b694e81138 (127.21.214.65). New cstate: current_term: 1 leader_uuid: "f8e3994507cd47bfa70535b694e81138" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8e3994507cd47bfa70535b694e81138" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 } health_report { overall_health: UNKNOWN } } }
W20260811 01:47:57.634438 6726 tablet.cc:2369] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:47:57.640793 6884 tablet.cc:2369] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:47:57.654310 7042 tablet.cc:2369] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:47:57.676707 7054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.671462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37648 (local address 127.21.214.65:36407)
0811 01:47:57.671675 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:57.671680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:57.671700 (+ 20us) server_negotiation.cc:407] Connection header received
0811 01:47:57.671801 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:57.671805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:57.671894 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:57.672023 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:47:57.672366 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.673086 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.674600 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.674992 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.675084 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:57.675581 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:47:57.676080 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:47:57.676219 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:57.676337 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 01:47:57.676406 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 01:47:57.685400 6831 raft_consensus.cc:1261] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555 [term 1 FOLLOWER]: Refusing update from remote peer f8e3994507cd47bfa70535b694e81138: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:47:57.685447 6986 raft_consensus.cc:1261] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522 [term 1 FOLLOWER]: Refusing update from remote peer f8e3994507cd47bfa70535b694e81138: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:47:57.686488 7072 consensus_queue.cc:1035] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d99554359d414dddbf1fe58a2c00a555" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 44947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:47:57.686908 7067 consensus_queue.cc:1035] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [LEADER]: Connected to new peer: Peer: permanent_uuid: "504a5cb11b0845a5a87ec23b38c6d522" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:47:57.716953 6512 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:52470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:47:57.717347 6512 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:52470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:47:57.721230 6512 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770: Sending DeleteTablet for 3 replicas of tablet 865ebd2dc2e74c3fa7cf4e4e41d42690
I20260811 01:47:57.722826 6807 tablet_service.cc:1514] Processing DeleteTablet for tablet 865ebd2dc2e74c3fa7cf4e4e41d42690 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:47:57 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:41496
I20260811 01:47:57.723035 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 6554
I20260811 01:47:57.723165 6652 tablet_service.cc:1514] Processing DeleteTablet for tablet 865ebd2dc2e74c3fa7cf4e4e41d42690 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:47:57 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:37636
I20260811 01:47:57.723902 6963 tablet_service.cc:1514] Processing DeleteTablet for tablet 865ebd2dc2e74c3fa7cf4e4e41d42690 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:47:57 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:40826
I20260811 01:47:57.724818 7091 tablet_replica.cc:330] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522: stopping tablet replica
I20260811 01:47:57.725301 7091 raft_consensus.cc:2229] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:47:57.725920 7091 raft_consensus.cc:2258] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:47:57.728880 7091 ts_tablet_manager.cc:1905] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:47:57.736995 7090 tablet_replica.cc:330] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138: stopping tablet replica
I20260811 01:47:57.737718 7090 raft_consensus.cc:2229] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:47:57.738534 7091 ts_tablet_manager.cc:1918] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:47:57.738598 7089 tablet_replica.cc:330] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555: stopping tablet replica
I20260811 01:47:57.738729 7091 log.cc:1196] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/865ebd2dc2e74c3fa7cf4e4e41d42690
I20260811 01:47:57.738938 7090 raft_consensus.cc:2258] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:47:57.739382 7091 ts_tablet_manager.cc:1939] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P 504a5cb11b0845a5a87ec23b38c6d522: Deleting consensus metadata
I20260811 01:47:57.739331 7089 raft_consensus.cc:2229] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:47:57.739918 7089 raft_consensus.cc:2258] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:47:57.741685 6498 catalog_manager.cc:4841] TS 504a5cb11b0845a5a87ec23b38c6d522 (127.21.214.67:38837): tablet 865ebd2dc2e74c3fa7cf4e4e41d42690 (table test-table [id=83a1e39d2ca2483ba66d2e0d23c9759a]) successfully deleted
I20260811 01:47:57.742415 7090 ts_tablet_manager.cc:1905] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:47:57.744982 7089 ts_tablet_manager.cc:1905] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:47:57.748886 7054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.725247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37656 (local address 127.21.214.65:36407)
0811 01:47:57.725454 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:57.725462 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:57.725487 (+ 25us) server_negotiation.cc:407] Connection header received
0811 01:47:57.725548 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:57.725552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:57.725623 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:57.725734 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:57.728174 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.729416 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.732097 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.733049 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.733225 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:57.738460 (+ 5235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:57.738482 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:57.738501 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:57.738552 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:57.743047 (+ 4495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:57.744739 (+ 1692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:57.744749 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:57.744755 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:57.745420 (+ 665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:57.745972 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:57.745977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:57.745981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:57.746460 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:57.746816 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:57.748158 (+ 1342us) server_negotiation.cc:299] Negotiation successful
0811 01:47:57.748451 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 01:47:57.749845 6719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:57.752091 7089 ts_tablet_manager.cc:1918] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:47:57.752270 7089 log.cc:1196] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/865ebd2dc2e74c3fa7cf4e4e41d42690
I20260811 01:47:57.752982 7089 ts_tablet_manager.cc:1939] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P d99554359d414dddbf1fe58a2c00a555: Deleting consensus metadata
I20260811 01:47:57.755214 6498 catalog_manager.cc:4841] TS d99554359d414dddbf1fe58a2c00a555 (127.21.214.66:44947): tablet 865ebd2dc2e74c3fa7cf4e4e41d42690 (table test-table [id=83a1e39d2ca2483ba66d2e0d23c9759a]) successfully deleted
I20260811 01:47:57.849133 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 6554
I20260811 01:47:57.849233 7090 ts_tablet_manager.cc:1918] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:47:57.880673 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 6729
W20260811 01:47:57.880762 6494 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.214.65:36407 (error 108)
I20260811 01:47:57.883244 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.882637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:36407 (local address 127.0.0.1:37660)
0811 01:47:57.882927 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:57.883047 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
W20260811 01:47:57.883519 6494 catalog_manager.cc:4584] TS f8e3994507cd47bfa70535b694e81138 (127.21.214.65:36407): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 865ebd2dc2e74c3fa7cf4e4e41d42690: Network error: Client connection negotiation failed: client connection to 127.21.214.65:36407: connect: Connection refused (error 111)
I20260811 01:47:57.900908 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.881991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41508 (local address 127.21.214.66:44947)
0811 01:47:57.882218 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:57.882225 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:57.882315 (+ 90us) server_negotiation.cc:407] Connection header received
0811 01:47:57.882480 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:57.882486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:57.882574 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:57.882701 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:57.884548 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.886054 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.888329 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:57.889247 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:57.889378 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:57.893081 (+ 3703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:57.893096 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:57.893116 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:57.893243 (+ 127us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:57.897416 (+ 4173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:57.898328 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:57.898340 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:57.898343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:57.898502 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:57.899003 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:57.899009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:57.899013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:57.899476 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:57.899838 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:57.900217 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 01:47:57.900480 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 01:47:57.901795 6878 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:57.937604 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.937049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:36407 (local address 127.0.0.1:37668)
0811 01:47:57.937326 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:57.937429 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260811 01:47:57.938786 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.938420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:36407 (local address 127.0.0.1:37672)
0811 01:47:57.938580 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:57.938669 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 01:47:57.994791 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 6729
I20260811 01:47:57.999402 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:57.998937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:36407 (local address 127.0.0.1:37674)
0811 01:47:57.999156 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:57.999255 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 01:47:58.000439 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:58.000076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:36407 (local address 127.0.0.1:37678)
0811 01:47:58.000234 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:58.000320 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 01:47:58.017087 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 6887
I20260811 01:47:58.032487 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:58.017966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40828 (local address 127.21.214.67:38837)
0811 01:47:58.018167 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 01:47:58.018175 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:47:58.018200 (+ 25us) server_negotiation.cc:407] Connection header received
0811 01:47:58.018463 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:47:58.018467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:47:58.018535 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:47:58.018645 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:47:58.020059 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:58.020832 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:58.022483 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:47:58.022918 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:47:58.023062 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:47:58.026023 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:47:58.026040 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:47:58.026059 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:47:58.026093 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:47:58.028886 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:58.029873 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:58.029885 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:58.029889 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:58.030106 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:47:58.030551 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:47:58.030555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:47:58.030557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:47:58.031146 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:47:58.031447 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 01:47:58.031831 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 01:47:58.032127 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 01:47:58.033279 7034 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:58.081341 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:58.080707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:36407 (local address 127.0.0.1:37692)
0811 01:47:58.080963 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:58.081069 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 01:47:58.082537 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:58.082166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:36407 (local address 127.0.0.1:37704)
0811 01:47:58.082360 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:58.082433 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260811 01:47:58.125423 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 6887
I20260811 01:47:58.149235 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 6476
I20260811 01:47:58.150712 6547 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:47:58.272686 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:58.272069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:36407 (local address 127.0.0.1:37708)
0811 01:47:58.272328 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:58.272438 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 01:47:58.272725 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 6476
I20260811 01:47:58.302163 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:36311
--webserver_interface=127.21.214.126
--webserver_port=38411
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:36311
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:47:58.661006 7100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:58.661413 7100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:58.661633 7100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:58.669267 7100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:47:58.669405 7100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:58.669510 7100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:58.669611 7100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:47:58.669704 7100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:47:58.680262 7100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:36311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:36311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=38411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:58.681885 7100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:58.683305 7100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:58.696266 7105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:58.696136 7108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:58.696136 7106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:58.697583 7100 server_base.cc:1034] running on GCE node
I20260811 01:47:58.698474 7100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:58.700021 7100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:58.701260 7100 hybrid_clock.cc:648] HybridClock initialized: now 1786412878701178 us; error 56 us; skew 500 ppm
I20260811 01:47:58.701584 7100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:58 dist-test-slave-9gcw krb5kdc[5842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412878, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:58.704970 7100 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:47:58.706624 7100 webserver.cc:460] Webserver started at http://127.21.214.126:38411/ using document root <none> and password file <none>
I20260811 01:47:58.707374 7100 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:58.707531 7100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:58.713379 7100 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 01:47:58.716426 7115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:58.717934 7100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 01:47:58.718122 7100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "4cc51b4cb4844ed8b52bb5752edd8770"
format_stamp: "Formatted at 2026-08-11 01:47:52 on dist-test-slave-9gcw"
server_key: "8f1e18d709bba2fb3e650fb36e0b3daa"
server_key_iv: "eef40a02dec53de7af4fbce3d8f76c08"
server_key_version: "encryptionkey@0"
I20260811 01:47:58.719216 7100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:58.744500 7100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:58.749094 7100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:58.749430 7100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:58.769757 7100 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:36311
I20260811 01:47:58.769769 7176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:36311 every 8 connection(s)
I20260811 01:47:58.771365 7100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:47:58.773008 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 7100
I20260811 01:47:58.774156 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:36407
--local_ip_for_outbound_sockets=127.21.214.65
--tserver_master_addrs=127.21.214.126:36311
--webserver_port=42509
--webserver_interface=127.21.214.65
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:47:58.782089 7177 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:47:58.787344 7177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770: Bootstrap starting.
I20260811 01:47:58.808964 7177 log.cc:824] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770: Log is configured to *not* fsync() on all Append() calls
I20260811 01:47:58.826490 7177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:47:58.827087 7177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770: Bootstrap complete.
I20260811 01:47:58.836849 7177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36311 } }
I20260811 01:47:58.837908 7177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cc51b4cb4844ed8b52bb5752edd8770, State: Initialized, Role: FOLLOWER
I20260811 01:47:58.838614 7177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36311 } }
I20260811 01:47:58.838871 7177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:47:58.839037 7177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:47:58.839229 7177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:47:58.843483 7177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36311 } }
I20260811 01:47:58.844074 7177 leader_election.cc:302] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cc51b4cb4844ed8b52bb5752edd8770; no voters:
I20260811 01:47:58.844808 7177 leader_election.cc:288] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 01:47:58.845809 7182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 01:47:58.846566 7182 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [term 3 LEADER]: Becoming Leader. State: Replica: 4cc51b4cb4844ed8b52bb5752edd8770, State: Running, Role: LEADER
I20260811 01:47:58.847113 7177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:47:58.847363 7182 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36311 } }
I20260811 01:47:58.850819 7186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cc51b4cb4844ed8b52bb5752edd8770. Latest consensus state: current_term: 3 leader_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36311 } } }
I20260811 01:47:58.851047 7185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc51b4cb4844ed8b52bb5752edd8770" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36311 } } }
I20260811 01:47:58.851318 7186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [sys.catalog]: This master's current role is: LEADER
I20260811 01:47:58.851405 7185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc51b4cb4844ed8b52bb5752edd8770 [sys.catalog]: This master's current role is: LEADER
I20260811 01:47:58.857610 7193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:47:58.867027 7193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:47:58.868791 7193 catalog_manager.cc:1259] Loaded cluster ID: 7b3df393207b46f4ab35f13706c35e33
I20260811 01:47:58.868958 7193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 01:47:58.871235 7193 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f97af8cb9ad google::LogMessage::Fail() at ??:0
@ 0x7f97af8ce88a google::LogMessage::SendToLog() at ??:0
@ 0x7f97af8cb4de google::LogMessage::Flush() at ??:0
@ 0x7f97af8cca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
I20260811 01:47:58.886392 7204 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f97c52fe512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f97c52fb976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f97c5383610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f97c5383439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f97b2517345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f97b251a91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f97b251a799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f97b24f5eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f97b57cf6db start_thread at ??:0
@ 0x7f97ae76561f clone at ??:0
W20260811 01:47:59.245116 7180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:59.245532 7180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:59.245644 7180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:59.254112 7180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:59.254240 7180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:59.254356 7180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:47:59.264782 7180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:36407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=42509
--tserver_master_addrs=127.21.214.126:36311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:59.266224 7180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:59.267652 7180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:59.282079 7229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:59.282155 7227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:59.282362 7226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:59.283145 7180 server_base.cc:1034] running on GCE node
I20260811 01:47:59.283974 7180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:59.285192 7180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:59.286485 7180 hybrid_clock.cc:648] HybridClock initialized: now 1786412879286412 us; error 62 us; skew 500 ppm
I20260811 01:47:59.286959 7180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:59 dist-test-slave-9gcw krb5kdc[5842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412879, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:59.290846 7180 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:47:59.292531 7180 webserver.cc:460] Webserver started at http://127.21.214.65:42509/ using document root <none> and password file <none>
I20260811 01:47:59.293481 7180 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:59.293684 7180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:59.300040 7180 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 01:47:59.303368 7236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:59.305286 7180 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 01:47:59.305487 7180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "f8e3994507cd47bfa70535b694e81138"
format_stamp: "Formatted at 2026-08-11 01:47:53 on dist-test-slave-9gcw"
server_key: "6aabc93bb9a434f5367fc3ac453b40d3"
server_key_iv: "0561acd35f4832eb3ea678cd25c69e04"
server_key_version: "encryptionkey@0"
I20260811 01:47:59.306588 7180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:59.323846 7180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:59.328332 7180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:59.328670 7180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:59.329926 7180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:47:59.331895 7245 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260811 01:47:59.338377 7245 ts_tablet_manager.cc:1852] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260811 01:47:59.339430 7245 ts_tablet_manager.cc:1905] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:47:59.347643 7245 ts_tablet_manager.cc:1918] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:47:59.347864 7245 log.cc:1196] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/865ebd2dc2e74c3fa7cf4e4e41d42690
I20260811 01:47:59.348313 7245 ts_tablet_manager.cc:1939] T 865ebd2dc2e74c3fa7cf4e4e41d42690 P f8e3994507cd47bfa70535b694e81138: Deleting consensus metadata
I20260811 01:47:59.348709 7180 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260811 01:47:59.348877 7180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.017s user 0.001s sys 0.000s
I20260811 01:47:59.349033 7180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:47:59.349130 7180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:59.386269 7180 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:36407
I20260811 01:47:59.386333 7372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:36407 every 8 connection(s)
I20260811 01:47:59.387858 7180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:47:59.393882 7375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:59.391406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:58231)
0811 01:47:59.392310 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:59.392637 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":399,"threads_started":1}
W20260811 01:47:59.394682 7373 heartbeater.cc:644] Failed to heartbeat to 127.21.214.126:36311 (0 consecutive failures): Network error: Failed to ping master at 127.21.214.126:36311: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
I20260811 01:47:59.395748 7375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:59.395304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:53741)
0811 01:47:59.395505 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:59.395606 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 01:47:59.396410 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 7180
I20260811 01:47:59.396811 7375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:59.396399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:50287)
0811 01:47:59.396576 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:59.396675 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 01:47:59.397056 7373 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 01:47:59.397312 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:44947
--local_ip_for_outbound_sockets=127.21.214.66
--tserver_master_addrs=127.21.214.126:36311
--webserver_port=41173
--webserver_interface=127.21.214.66
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:47:59.769603 7376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:47:59.769944 7376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:47:59.770130 7376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:47:59.778775 7376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:47:59.778921 7376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:47:59.779148 7376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:47:59.790860 7376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:44947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=41173
--tserver_master_addrs=127.21.214.126:36311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:47:59.792459 7376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:47:59.794055 7376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:47:59.810086 7381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:59.810035 7382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:47:59.809991 7384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:47:59.811164 7376 server_base.cc:1034] running on GCE node
I20260811 01:47:59.811967 7376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:47:59.813079 7376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:47:59.814360 7376 hybrid_clock.cc:648] HybridClock initialized: now 1786412879814300 us; error 49 us; skew 500 ppm
I20260811 01:47:59.814746 7376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:47:59 dist-test-slave-9gcw krb5kdc[5842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412879, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:47:59.818677 7376 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:47:59.820328 7376 webserver.cc:460] Webserver started at http://127.21.214.66:41173/ using document root <none> and password file <none>
I20260811 01:47:59.821123 7376 fs_manager.cc:359] Metadata directory not provided
I20260811 01:47:59.821354 7376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:47:59.827628 7376 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 01:47:59.830942 7394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:59.832410 7376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 01:47:59.832602 7376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "d99554359d414dddbf1fe58a2c00a555"
format_stamp: "Formatted at 2026-08-11 01:47:53 on dist-test-slave-9gcw"
server_key: "62fa4db0ab39d45910c0c6732cbe574d"
server_key_iv: "6ae26b2020fdbe122c69674de6d7d900"
server_key_version: "encryptionkey@0"
I20260811 01:47:59.833755 7376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:47:59.846134 7376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:47:59.851025 7376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:47:59.851353 7376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:47:59.852453 7376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:47:59.853971 7376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:47:59.854096 7376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:59.854242 7376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:47:59.854344 7376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:47:59.891073 7376 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:44947
I20260811 01:47:59.891170 7527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:44947 every 8 connection(s)
I20260811 01:47:59.892702 7376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:47:59.898355 7531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:59.896156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:40237)
0811 01:47:59.896969 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:59.897402 (+ 433us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":306,"threads_started":1}
W20260811 01:47:59.899152 7528 heartbeater.cc:644] Failed to heartbeat to 127.21.214.126:36311 (0 consecutive failures): Network error: Failed to ping master at 127.21.214.126:36311: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
I20260811 01:47:59.899595 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 7376
I20260811 01:47:59.900172 7531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:59.899772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:56033)
0811 01:47:59.899932 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:59.900015 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 01:47:59.900319 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:38837
--local_ip_for_outbound_sockets=127.21.214.67
--tserver_master_addrs=127.21.214.126:36311
--webserver_port=33789
--webserver_interface=127.21.214.67
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:47:59.901531 7531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:47:59.900749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:34925)
0811 01:47:59.901270 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 01:47:59.901395 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":441,"threads_started":1}
W20260811 01:47:59.901769 7528 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 01:48:00.272693 7533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:48:00.273028 7533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:48:00.273268 7533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:48:00.281090 7533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:48:00.281311 7533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:48:00.281539 7533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:48:00.292382 7533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:38837
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=33789
--tserver_master_addrs=127.21.214.126:36311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:48:00.294006 7533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:48:00.295594 7533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:48:00.311434 7539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:48:00.311472 7545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:48:00.311408 7541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:48:00.312453 7533 server_base.cc:1034] running on GCE node
I20260811 01:48:00.313124 7533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:48:00.314273 7533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:48:00.315542 7533 hybrid_clock.cc:648] HybridClock initialized: now 1786412880315482 us; error 59 us; skew 500 ppm
I20260811 01:48:00.315927 7533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:48:00 dist-test-slave-9gcw krb5kdc[5842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412880, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:48:00.319478 7533 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:48:00.321066 7533 webserver.cc:460] Webserver started at http://127.21.214.67:33789/ using document root <none> and password file <none>
I20260811 01:48:00.321863 7533 fs_manager.cc:359] Metadata directory not provided
I20260811 01:48:00.321977 7533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:48:00.327780 7533 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 01:48:00.331203 7554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:00.332914 7533 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260811 01:48:00.333142 7533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "504a5cb11b0845a5a87ec23b38c6d522"
format_stamp: "Formatted at 2026-08-11 01:47:54 on dist-test-slave-9gcw"
server_key: "705e706b8bea8f0e6550536b71fbfb0b"
server_key_iv: "24249dc36759d3be667e1f20ecffe0b4"
server_key_version: "encryptionkey@0"
I20260811 01:48:00.334352 7533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:48:00.381217 7533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:48:00.386193 7533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:48:00.386503 7533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:48:00.387603 7533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:48:00.389087 7533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:48:00.389250 7533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 01:48:00.389379 7533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:48:00.389490 7533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:00.399080 7585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:00.397928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:43297)
0811 01:48:00.398529 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:00.398661 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":262,"threads_started":1}
I20260811 01:48:00.426595 7533 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:38837
I20260811 01:48:00.426785 7683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:38837 every 8 connection(s)
I20260811 01:48:00.428146 7533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:48:00.433691 7686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:00.431549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:52625)
0811 01:48:00.432385 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:00.432681 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":330,"threads_started":1}
I20260811 01:48:00.434092 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 7533
W20260811 01:48:00.434417 7684 heartbeater.cc:644] Failed to heartbeat to 127.21.214.126:36311 (0 consecutive failures): Network error: Failed to ping master at 127.21.214.126:36311: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
I20260811 01:48:00.435308 7686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:00.434957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:48877)
0811 01:48:00.435119 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:00.435200 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 01:48:00.436236 7686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:00.435911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:37557)
0811 01:48:00.436051 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:00.436134 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 01:48:00.436573 7684 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 01:48:00.903424 7689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:00.902498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:50145)
0811 01:48:00.903022 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:00.903156 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":252,"threads_started":1}
I20260811 01:48:01.400877 7690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:01.400098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:53135)
0811 01:48:01.400611 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:01.400707 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":279,"threads_started":1}
I20260811 01:48:01.438511 7692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:01.437285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:42725)
0811 01:48:01.437886 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:01.438081 (+ 195us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":265,"threads_started":1}
I20260811 01:48:01.905339 7693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:01.904548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:49381)
0811 01:48:01.905020 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:01.905114 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":242,"threads_started":1}
I20260811 01:48:02.403091 7694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:02.402018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:43149)
0811 01:48:02.402797 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:02.402904 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":446,"threads_started":1}
I20260811 01:48:02.440261 7695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:02.439530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:33383)
0811 01:48:02.440000 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:02.440085 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":247,"threads_started":1}
I20260811 01:48:02.907461 7696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:02.906495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:48845)
0811 01:48:02.907092 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:02.907191 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":349,"threads_started":1}
I20260811 01:48:03.405078 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:03.404248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:44693)
0811 01:48:03.404798 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:03.404904 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":281,"threads_started":1}
I20260811 01:48:03.442222 7699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:03.441423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:56351)
0811 01:48:03.441920 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:03.442023 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":244,"threads_started":1}
I20260811 01:48:03.909502 7700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:03.908632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:50597)
0811 01:48:03.909115 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:03.909302 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":228,"threads_started":1}
I20260811 01:48:04.407160 7701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:04.406255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:36037)
0811 01:48:04.406881 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:04.406971 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":383,"threads_started":1}
I20260811 01:48:04.444026 7702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:04.443303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:59895)
0811 01:48:04.443762 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:04.443848 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":235,"threads_started":1}
I20260811 01:48:04.911478 7704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:04.910566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:59307)
0811 01:48:04.911180 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:04.911296 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":290,"threads_started":1}
I20260811 01:48:05.409392 7705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:05.408367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:45309)
0811 01:48:05.408985 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:05.409098 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":328,"threads_started":1}
I20260811 01:48:05.446336 7706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:05.445295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:47815)
0811 01:48:05.446018 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:05.446147 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":453,"threads_started":1}
I20260811 01:48:05.913466 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:05.912575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:39869)
0811 01:48:05.913113 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:05.913290 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":274,"threads_started":1}
I20260811 01:48:06.411628 7709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:06.410585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:46565)
0811 01:48:06.411222 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:06.411327 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":333,"threads_started":1}
I20260811 01:48:06.448467 7710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:06.447479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:60031)
0811 01:48:06.448207 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:06.448305 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":496,"threads_started":1}
I20260811 01:48:06.915472 7711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:06.914572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:32861)
0811 01:48:06.915130 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:06.915231 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":237,"threads_started":1}
I20260811 01:48:07.413368 7712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:07.412597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:40953)
0811 01:48:07.413078 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:07.413196 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":256,"threads_started":1}
I20260811 01:48:07.450726 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:07.449814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:47169)
0811 01:48:07.450350 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:07.450447 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":240,"threads_started":1}
I20260811 01:48:07.917464 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:07.916567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:59891)
0811 01:48:07.917111 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:07.917293 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":292,"threads_started":1}
I20260811 01:48:08.415158 7716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:08.414351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:45957)
0811 01:48:08.414870 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:08.414976 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":252,"threads_started":1}
I20260811 01:48:08.452517 7717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:08.451779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:44887)
0811 01:48:08.452264 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:08.452354 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":253,"threads_started":1}
I20260811 01:48:08.919328 7718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:08.918549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:53503)
0811 01:48:08.919039 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:08.919131 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":257,"threads_started":1}
I20260811 01:48:09.416999 7719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:09.416224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:35385)
0811 01:48:09.416733 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:09.416828 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":255,"threads_started":1}
I20260811 01:48:09.454627 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:09.453724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:46615)
0811 01:48:09.454350 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:09.454437 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":363,"threads_started":1}
I20260811 01:48:09.921195 7721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:09.920383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:58207)
0811 01:48:09.920900 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:09.920981 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":296,"threads_started":1}
I20260811 01:48:10.419087 7722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:10.418133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:55827)
0811 01:48:10.418780 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:10.418882 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":349,"threads_started":1}
I20260811 01:48:10.456499 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:10.455692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:35299)
0811 01:48:10.456250 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:10.456331 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":338,"threads_started":1}
I20260811 01:48:10.923467 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:10.922373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:34965)
0811 01:48:10.923046 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:10.923175 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":375,"threads_started":1}
I20260811 01:48:11.421094 7726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:11.420313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:56183)
0811 01:48:11.420833 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:11.420926 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":288,"threads_started":1}
I20260811 01:48:11.458460 7727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:11.457604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:39229)
0811 01:48:11.458134 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:11.458264 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":280,"threads_started":1}
I20260811 01:48:11.925467 7728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:11.924665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:45313)
0811 01:48:11.925112 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:11.925309 (+ 197us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":217,"threads_started":1}
I20260811 01:48:12.423113 7729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:12.422227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:52979)
0811 01:48:12.422802 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:12.422910 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":304,"threads_started":1}
I20260811 01:48:12.460407 7730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:12.459572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:59727)
0811 01:48:12.460155 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:12.460240 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":350,"threads_started":1}
I20260811 01:48:12.927640 7731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:12.926492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:44975)
0811 01:48:12.927187 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:12.927343 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":335,"threads_started":1}
I20260811 01:48:13.424858 7732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:13.424103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:51571)
0811 01:48:13.424554 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:13.424642 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":234,"threads_started":1}
I20260811 01:48:13.462437 7735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:13.461551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:33173)
0811 01:48:13.462133 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:13.462240 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":314,"threads_started":1}
I20260811 01:48:13.929458 7736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:13.928651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:40939)
0811 01:48:13.929118 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:13.929238 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":241,"threads_started":1}
I20260811 01:48:14.427224 7737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:14.426337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:45135)
0811 01:48:14.426916 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:14.427039 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":306,"threads_started":1}
I20260811 01:48:14.464365 7738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:14.463600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:60711)
0811 01:48:14.464088 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:14.464177 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":255,"threads_started":1}
I20260811 01:48:14.931594 7739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:14.930742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.66:53837)
0811 01:48:14.931278 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:14.931391 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":288,"threads_started":1}
I20260811 01:48:15.429266 7741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:15.428383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.65:50837)
0811 01:48:15.428919 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:15.429030 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":241,"threads_started":1}
I20260811 01:48:15.435734 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 7180
I20260811 01:48:15.456940 7742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:15.437048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33904 (local address 127.21.214.65:36407)
0811 01:48:15.438031 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:15.438050 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:15.438119 (+ 69us) server_negotiation.cc:407] Connection header received
0811 01:48:15.440777 (+ 2658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:15.440789 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:15.441334 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:15.441676 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:15.443421 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:15.444831 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:15.446304 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:15.447022 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:15.447163 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:15.450215 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:15.450225 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:15.450244 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:15.450290 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:15.453473 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:15.454224 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:15.454231 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:15.454240 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:15.454422 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:15.454842 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:15.454845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:15.454847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:15.455429 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:15.455675 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:15.456113 (+ 438us) server_negotiation.cc:299] Negotiation successful
0811 01:48:15.456585 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":761,"thread_start_us":591,"threads_started":1}
I20260811 01:48:15.458524 7367 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:48:15.466632 7745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:15.465793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:36311 (local address 127.21.214.67:56017)
0811 01:48:15.466318 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:15.466444 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:36311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":275,"threads_started":1}
I20260811 01:48:15.549170 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 7180
I20260811 01:48:15.575392 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 7376
I20260811 01:48:15.596235 7747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:15.576532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47216 (local address 127.21.214.66:44947)
0811 01:48:15.577096 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:15.577111 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:15.577240 (+ 129us) server_negotiation.cc:407] Connection header received
0811 01:48:15.578803 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:15.578812 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:15.579274 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:15.579662 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:15.581401 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:15.582713 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:15.584224 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:15.584991 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:15.585257 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:15.588487 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:15.588495 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:15.588514 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:15.588560 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:15.591979 (+ 3419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:15.592992 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:15.593001 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:15.593012 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:15.593310 (+ 298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:15.593752 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:15.593758 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:15.593761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:15.594415 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:15.594785 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:15.595292 (+ 507us) server_negotiation.cc:299] Negotiation successful
0811 01:48:15.595912 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":289,"threads_started":1}
I20260811 01:48:15.597829 7523 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:48:15.686223 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 7376
I20260811 01:48:15.710352 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 7533
I20260811 01:48:15.731159 7750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:15.711795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54314 (local address 127.21.214.67:38837)
0811 01:48:15.712438 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:15.712453 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:15.712497 (+ 44us) server_negotiation.cc:407] Connection header received
0811 01:48:15.714209 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:15.714219 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:15.714681 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:15.714995 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:15.716554 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:15.718044 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:15.719572 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:15.720256 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:15.720401 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:15.723766 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:15.723774 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:15.723793 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:15.723835 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:15.727132 (+ 3297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:15.728012 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:15.728020 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:15.728029 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:15.728280 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:15.728729 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:15.728733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:15.728736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:15.729543 (+ 807us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:15.729853 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:15.730395 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 01:48:15.730833 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":321,"threads_started":1}
I20260811 01:48:15.732921 7678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:48:15.818881 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 7533
2026-08-11T01:48:15Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25319 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:48:15 dist-test-slave-9gcw krb5kdc[7756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:48:15 dist-test-slave-9gcw krb5kdc[7756](info): set up 2 sockets
Aug 11 01:48:15 dist-test-slave-9gcw krb5kdc[7756](info): commencing operation
krb5kdc: starting...
W20260811 01:48:17.111162 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.003s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:48:17 dist-test-slave-9gcw krb5kdc[7756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:48:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:48:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:48:17.435321 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:40013
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:40013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:48:17.800688 7771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:48:17.801020 7771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:48:17.801256 7771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:48:17.809453 7771 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 01:48:17.809566 7771 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 01:48:17.809645 7771 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 01:48:17.809731 7771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:48:17.809834 7771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:48:17.809937 7771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:48:17.810026 7771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:48:17.810105 7771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:48:17.820788 7771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:40013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:40013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:48:17.822345 7771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:48:17.823812 7771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:48:17.837504 7776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:48:17.837486 7777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:48:17.837604 7779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:48:17.838788 7771 server_base.cc:1034] running on GCE node
I20260811 01:48:17.839691 7771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:48:17.841408 7771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:48:17.842753 7771 hybrid_clock.cc:648] HybridClock initialized: now 1786412897842620 us; error 45 us; skew 500 ppm
I20260811 01:48:17.843134 7771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:48:17 dist-test-slave-9gcw krb5kdc[7756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412897, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:48:17.852511 7771 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:48:17.854928 7771 webserver.cc:460] Webserver started at http://127.21.214.126:35031/ using document root <none> and password file <none>
I20260811 01:48:17.855806 7771 fs_manager.cc:359] Metadata directory not provided
I20260811 01:48:17.855958 7771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:48:17.856323 7771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:48:17.859764 7771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "fc0a298e73dd4848ae307d38c888ded9"
format_stamp: "Formatted at 2026-08-11 01:48:17 on dist-test-slave-9gcw"
server_key: "10087ef2ea7922aab8b3efddba6985ef"
server_key_iv: "317d1cde71154733a98094111cb40be4"
server_key_version: "encryptionkey@0"
I20260811 01:48:17.860656 7771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "fc0a298e73dd4848ae307d38c888ded9"
format_stamp: "Formatted at 2026-08-11 01:48:17 on dist-test-slave-9gcw"
server_key: "10087ef2ea7922aab8b3efddba6985ef"
server_key_iv: "317d1cde71154733a98094111cb40be4"
server_key_version: "encryptionkey@0"
I20260811 01:48:17.866698 7771 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.006s
I20260811 01:48:17.870975 7788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:17.872710 7771 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260811 01:48:17.872984 7771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "fc0a298e73dd4848ae307d38c888ded9"
format_stamp: "Formatted at 2026-08-11 01:48:17 on dist-test-slave-9gcw"
server_key: "10087ef2ea7922aab8b3efddba6985ef"
server_key_iv: "317d1cde71154733a98094111cb40be4"
server_key_version: "encryptionkey@0"
I20260811 01:48:17.873387 7771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:48:17.891790 7771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:48:17.896692 7771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:48:17.897001 7771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:48:17.917707 7771 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:40013
I20260811 01:48:17.917729 7848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:40013 every 8 connection(s)
I20260811 01:48:17.919162 7771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:48:17.923606 7849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:48:17.928336 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 7771
I20260811 01:48:17.928577 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:48:17.929467 22361 external_mini_cluster.cc:1428] Setting key 3a2254d8c05308809299c5f79043afc5
I20260811 01:48:17.936195 7849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: Bootstrap starting.
I20260811 01:48:17.939633 7849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: Neither blocks nor log segments found. Creating new log.
I20260811 01:48:17.940750 7849 log.cc:824] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: Log is configured to *not* fsync() on all Append() calls
I20260811 01:48:17.943981 7849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: No bootstrap required, opened a new log
Aug 11 01:48:17 dist-test-slave-9gcw krb5kdc[7756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:48:17.953699 7849 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0a298e73dd4848ae307d38c888ded9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40013 } }
I20260811 01:48:17.954452 7849 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:48:17.954587 7849 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0a298e73dd4848ae307d38c888ded9, State: Initialized, Role: FOLLOWER
I20260811 01:48:17.955228 7849 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0a298e73dd4848ae307d38c888ded9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40013 } }
I20260811 01:48:17.955576 7849 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:48:17.955781 7853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:17.931859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38960 (local address 127.21.214.126:40013)
0811 01:48:17.932812 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:17.932825 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:17.932873 (+ 48us) server_negotiation.cc:407] Connection header received
0811 01:48:17.934791 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:17.934817 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:17.935245 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:17.935680 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:17.937457 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:17.938860 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:17.940334 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:17.941104 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:17.941341 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:17.946318 (+ 4977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:17.946329 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:17.946350 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:17.946401 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:17.950437 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:17.951463 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:17.951471 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:17.951478 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:17.951680 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:17.952197 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:17.952201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:17.952203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:17.952863 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:17.953096 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:17.953783 (+ 687us) server_negotiation.cc:299] Negotiation successful
0811 01:48:17.954303 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":404,"threads_started":1}
I20260811 01:48:17.955761 7849 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:48:17.956051 7849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:48:17.958770 7849 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0a298e73dd4848ae307d38c888ded9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40013 } }
I20260811 01:48:17.959386 7849 leader_election.cc:302] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc0a298e73dd4848ae307d38c888ded9; no voters:
I20260811 01:48:17.960094 7849 leader_election.cc:288] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:48:17.960273 7855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:48:17.961040 7855 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 1 LEADER]: Becoming Leader. State: Replica: fc0a298e73dd4848ae307d38c888ded9, State: Running, Role: LEADER
I20260811 01:48:17.961745 7855 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0a298e73dd4848ae307d38c888ded9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40013 } }
I20260811 01:48:17.962559 7849 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:48:17.965205 7856 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc0a298e73dd4848ae307d38c888ded9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0a298e73dd4848ae307d38c888ded9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40013 } } }
I20260811 01:48:17.965317 7857 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc0a298e73dd4848ae307d38c888ded9. Latest consensus state: current_term: 1 leader_uuid: "fc0a298e73dd4848ae307d38c888ded9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0a298e73dd4848ae307d38c888ded9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40013 } } }
I20260811 01:48:17.965643 7856 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [sys.catalog]: This master's current role is: LEADER
I20260811 01:48:17.965726 7857 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [sys.catalog]: This master's current role is: LEADER
I20260811 01:48:17.966653 7864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:48:17.974009 7864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:48:17.984043 7864 catalog_manager.cc:1347] Generated new cluster ID: d7f0f9cd8c9a43daa9374df302479ef7
I20260811 01:48:17.984202 7864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:48:18.004985 7864 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:48:18.006515 7864 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:48:18.016510 7864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: Generated new TSK 0
I20260811 01:48:18.017565 7864 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
I20260811 01:48:18.114209 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40013
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:48:18.517135 7879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:48:18.517568 7879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:48:18.517670 7879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:48:18.525213 7879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:48:18.525367 7879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:48:18.525487 7879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:48:18.535733 7879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:48:18.537287 7879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:48:18.538678 7879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:48:18.553905 7884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:48:18.553905 7885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:48:18.553838 7888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:48:18.555028 7879 server_base.cc:1034] running on GCE node
I20260811 01:48:18.555750 7879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:48:18.556814 7879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:48:18.558030 7879 hybrid_clock.cc:648] HybridClock initialized: now 1786412898557928 us; error 95 us; skew 500 ppm
I20260811 01:48:18.558368 7879 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:48:18 dist-test-slave-9gcw krb5kdc[7756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412898, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:48:18.561803 7879 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:48:18.563619 7879 webserver.cc:460] Webserver started at http://127.21.214.65:46845/ using document root <none> and password file <none>
I20260811 01:48:18.564493 7879 fs_manager.cc:359] Metadata directory not provided
I20260811 01:48:18.564651 7879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:48:18.564980 7879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:48:18.568472 7879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "c86dbfd773a14e1181737ec35c0fc554"
format_stamp: "Formatted at 2026-08-11 01:48:18 on dist-test-slave-9gcw"
server_key: "25260aa86b4b07b8d9c35cd5bf02c4c9"
server_key_iv: "954e7a3c2d74fb2d465c9a0079c9aa28"
server_key_version: "encryptionkey@0"
I20260811 01:48:18.569583 7879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "c86dbfd773a14e1181737ec35c0fc554"
format_stamp: "Formatted at 2026-08-11 01:48:18 on dist-test-slave-9gcw"
server_key: "25260aa86b4b07b8d9c35cd5bf02c4c9"
server_key_iv: "954e7a3c2d74fb2d465c9a0079c9aa28"
server_key_version: "encryptionkey@0"
I20260811 01:48:18.575627 7879 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.005s
I20260811 01:48:18.580169 7896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:18.581948 7879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 01:48:18.582178 7879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "c86dbfd773a14e1181737ec35c0fc554"
format_stamp: "Formatted at 2026-08-11 01:48:18 on dist-test-slave-9gcw"
server_key: "25260aa86b4b07b8d9c35cd5bf02c4c9"
server_key_iv: "954e7a3c2d74fb2d465c9a0079c9aa28"
server_key_version: "encryptionkey@0"
I20260811 01:48:18.582502 7879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:48:18.604056 7879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:48:18.609542 7879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:48:18.609848 7879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:48:18.610966 7879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:48:18.612457 7879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:48:18.612545 7879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:18.612641 7879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:48:18.612694 7879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:18.650384 7879 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:40993
I20260811 01:48:18.650450 8023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:40993 every 8 connection(s)
I20260811 01:48:18.651794 7879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:48:18.654508 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 7879
I20260811 01:48:18.654773 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:48:18.655781 22361 external_mini_cluster.cc:1428] Setting key 0f0c208241612d92f3e976ff9528eee3
Aug 11 01:48:18 dist-test-slave-9gcw krb5kdc[7756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412898, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:48:18.677256 8026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:18.654897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:51585 (local address 127.21.214.126:40013)
0811 01:48:18.655379 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:18.655385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:18.656842 (+ 1457us) server_negotiation.cc:407] Connection header received
0811 01:48:18.659284 (+ 2442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:18.659293 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:18.659395 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:18.659525 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:18.661231 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:18.662483 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:18.664735 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:18.665207 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:18.665393 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:18.669814 (+ 4421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:18.669821 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:18.669827 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:18.669877 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:18.673699 (+ 3822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:18.674574 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:18.674581 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:18.674585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:18.674731 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:18.675226 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:18.675231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:18.675234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:18.675567 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:18.675728 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:18.676552 (+ 824us) server_negotiation.cc:299] Negotiation successful
0811 01:48:18.676802 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":254,"threads_started":1}
I20260811 01:48:18.678490 8027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:18.655673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:51585)
0811 01:48:18.656600 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:18.656659 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 01:48:18.658959 (+ 2300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:48:18.659779 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:48:18.659788 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:48:18.660382 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:48:18.661005 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:18.661024 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:18.662740 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:18.662749 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:18.664488 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:18.664503 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:18.665503 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:18.665513 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:18.665804 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:18.666600 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:48:18.666625 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:48:18.669509 (+ 2884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:48:18.673886 (+ 4377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:48:18.673898 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:48:18.673926 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:48:18.674427 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:48:18.674865 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:48:18.674870 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:48:18.674876 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:48:18.675096 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:48:18.675729 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:48:18.675740 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:48:18.676140 (+ 400us) client_negotiation.cc:769] Sending connection context
0811 01:48:18.676453 (+ 313us) client_negotiation.cc:240] Negotiation successful
0811 01:48:18.677084 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":382,"threads_started":1}
I20260811 01:48:18.680321 8024 heartbeater.cc:347] Connected to a master server at 127.21.214.126:40013
I20260811 01:48:18.680740 8024 heartbeater.cc:464] Registering TS with master...
I20260811 01:48:18.681789 8024 heartbeater.cc:511] Master 127.21.214.126:40013 requested a full tablet report, sending...
I20260811 01:48:18.684638 7806 ts_manager.cc:194] Registered new tserver with Master: c86dbfd773a14e1181737ec35c0fc554 (127.21.214.65:40993)
I20260811 01:48:18.686288 7806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:51585
WARNING: no policy specified for kudu/127.21.214.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
I20260811 01:48:18.730507 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40013
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:48:19.127628 8030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:48:19.128000 8030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:48:19.128122 8030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:48:19.136602 8030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:48:19.136761 8030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:48:19.136893 8030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:48:19.148675 8030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:48:19.150235 8030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:48:19.151716 8030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:48:19.168067 8038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:48:19.168345 8037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:48:19.168548 8040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:48:19.169193 8030 server_base.cc:1034] running on GCE node
I20260811 01:48:19.169934 8030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:48:19.171110 8030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:48:19.172384 8030 hybrid_clock.cc:648] HybridClock initialized: now 1786412899172319 us; error 59 us; skew 500 ppm
I20260811 01:48:19.172801 8030 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:48:19 dist-test-slave-9gcw krb5kdc[7756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412899, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:48:19.176501 8030 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:48:19.178439 8030 webserver.cc:460] Webserver started at http://127.21.214.66:36197/ using document root <none> and password file <none>
I20260811 01:48:19.179334 8030 fs_manager.cc:359] Metadata directory not provided
I20260811 01:48:19.179495 8030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:48:19.179836 8030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:48:19.183154 8030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "27c87199ae444777be34b8ce818f1ac2"
format_stamp: "Formatted at 2026-08-11 01:48:19 on dist-test-slave-9gcw"
server_key: "44df448c0f3782a0c495cf278822c28a"
server_key_iv: "252f47ffff186787770c7efc623d64af"
server_key_version: "encryptionkey@0"
I20260811 01:48:19.184052 8030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "27c87199ae444777be34b8ce818f1ac2"
format_stamp: "Formatted at 2026-08-11 01:48:19 on dist-test-slave-9gcw"
server_key: "44df448c0f3782a0c495cf278822c28a"
server_key_iv: "252f47ffff186787770c7efc623d64af"
server_key_version: "encryptionkey@0"
I20260811 01:48:19.190408 8030 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 01:48:19.194545 8047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:19.196218 8030 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 01:48:19.196432 8030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "27c87199ae444777be34b8ce818f1ac2"
format_stamp: "Formatted at 2026-08-11 01:48:19 on dist-test-slave-9gcw"
server_key: "44df448c0f3782a0c495cf278822c28a"
server_key_iv: "252f47ffff186787770c7efc623d64af"
server_key_version: "encryptionkey@0"
I20260811 01:48:19.196686 8030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:48:19.211905 8030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:48:19.216837 8030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:48:19.217201 8030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:48:19.218298 8030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:48:19.219760 8030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:48:19.219877 8030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:19.220041 8030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:48:19.220117 8030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:19.258126 8030 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:40303
I20260811 01:48:19.258179 8187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:40303 every 8 connection(s)
I20260811 01:48:19.259531 8030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:48:19.264892 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 8030
I20260811 01:48:19.265215 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:48:19.265865 22361 external_mini_cluster.cc:1428] Setting key 6ef56ea6251da88aeebfe50da208e8a0
Aug 11 01:48:19 dist-test-slave-9gcw krb5kdc[7756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412899, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:48:19.285698 8190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:19.262610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:54227 (local address 127.21.214.126:40013)
0811 01:48:19.263392 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:19.263402 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:19.264820 (+ 1418us) server_negotiation.cc:407] Connection header received
0811 01:48:19.267495 (+ 2675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:19.267500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:19.267604 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:19.267718 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:19.270208 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.270996 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.273726 (+ 2730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.274240 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.274351 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:19.279209 (+ 4858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:19.279214 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:19.279218 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:19.279263 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:19.282193 (+ 2930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:19.283070 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:19.283074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:19.283077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:19.283207 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:19.283740 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:19.283746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:19.283748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:19.284039 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:19.284205 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:19.284982 (+ 777us) server_negotiation.cc:299] Negotiation successful
0811 01:48:19.285315 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":436,"threads_started":1}
I20260811 01:48:19.286863 8191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:19.263458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:54227)
0811 01:48:19.264499 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:19.264563 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 01:48:19.267189 (+ 2626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:48:19.268027 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:48:19.268036 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:48:19.268928 (+ 892us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:48:19.269970 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:19.270004 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.271191 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.271200 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:19.273528 (+ 2328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:19.273547 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.274500 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.274510 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:19.274903 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:19.275762 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:48:19.275802 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:48:19.278906 (+ 3104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:48:19.282380 (+ 3474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:48:19.282391 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:48:19.282413 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:48:19.282911 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:48:19.283375 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:48:19.283380 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:48:19.283384 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:48:19.283596 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:48:19.284178 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:48:19.284187 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:48:19.284575 (+ 388us) client_negotiation.cc:769] Sending connection context
0811 01:48:19.284870 (+ 295us) client_negotiation.cc:240] Negotiation successful
0811 01:48:19.285512 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":313,"threads_started":1}
I20260811 01:48:19.288898 8188 heartbeater.cc:347] Connected to a master server at 127.21.214.126:40013
I20260811 01:48:19.289304 8188 heartbeater.cc:464] Registering TS with master...
I20260811 01:48:19.290095 8188 heartbeater.cc:511] Master 127.21.214.126:40013 requested a full tablet report, sending...
I20260811 01:48:19.291893 7806 ts_manager.cc:194] Registered new tserver with Master: 27c87199ae444777be34b8ce818f1ac2 (127.21.214.66:40303)
I20260811 01:48:19.292773 7806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:54227
WARNING: no policy specified for kudu/127.21.214.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
I20260811 01:48:19.340669 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40013
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:48:19.690477 8024 heartbeater.cc:503] Master 127.21.214.126:40013 was elected leader, sending a full tablet report...
W20260811 01:48:19.722280 8195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:48:19.722612 8195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:48:19.722787 8195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:48:19.730512 8195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:48:19.730655 8195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:48:19.730842 8195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:48:19.741508 8195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:48:19.743032 8195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:48:19.744575 8195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:48:19.759434 8200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:48:19.759678 8201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:48:19.759559 8203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:48:19.760913 8195 server_base.cc:1034] running on GCE node
I20260811 01:48:19.761674 8195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:48:19.762816 8195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:48:19.764029 8195 hybrid_clock.cc:648] HybridClock initialized: now 1786412899763978 us; error 39 us; skew 500 ppm
I20260811 01:48:19.764400 8195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:48:19 dist-test-slave-9gcw krb5kdc[7756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412899, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:48:19.767884 8195 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:48:19.769634 8195 webserver.cc:460] Webserver started at http://127.21.214.67:39387/ using document root <none> and password file <none>
I20260811 01:48:19.770364 8195 fs_manager.cc:359] Metadata directory not provided
I20260811 01:48:19.770475 8195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:48:19.770762 8195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:48:19.774055 8195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "704161483bfb45e2a571fa19ce99e7f4"
format_stamp: "Formatted at 2026-08-11 01:48:19 on dist-test-slave-9gcw"
server_key: "82a8861f769c6a7f85d1b337619c2497"
server_key_iv: "a1097fee717e69b6e422853bf81a3181"
server_key_version: "encryptionkey@0"
I20260811 01:48:19.774888 8195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "704161483bfb45e2a571fa19ce99e7f4"
format_stamp: "Formatted at 2026-08-11 01:48:19 on dist-test-slave-9gcw"
server_key: "82a8861f769c6a7f85d1b337619c2497"
server_key_iv: "a1097fee717e69b6e422853bf81a3181"
server_key_version: "encryptionkey@0"
I20260811 01:48:19.780947 8195 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 01:48:19.785388 8212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:19.787241 8195 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 01:48:19.787487 8195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "704161483bfb45e2a571fa19ce99e7f4"
format_stamp: "Formatted at 2026-08-11 01:48:19 on dist-test-slave-9gcw"
server_key: "82a8861f769c6a7f85d1b337619c2497"
server_key_iv: "a1097fee717e69b6e422853bf81a3181"
server_key_version: "encryptionkey@0"
I20260811 01:48:19.787747 8195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:48:19.819936 8195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:48:19.825112 8195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:48:19.825430 8195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:48:19.826490 8195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:48:19.827968 8195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:48:19.828045 8195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:19.828136 8195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:48:19.828182 8195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:19.865531 8195 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:40965
I20260811 01:48:19.865572 8342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:40965 every 8 connection(s)
I20260811 01:48:19.867097 8195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:48:19.867446 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 8195
I20260811 01:48:19.867673 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:48:19.868295 22361 external_mini_cluster.cc:1428] Setting key a882ac355cb64055affb991d4bb60ebd
Aug 11 01:48:19 dist-test-slave-9gcw krb5kdc[7756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412899, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:48:19.892843 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:19.870549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:43805 (local address 127.21.214.126:40013)
0811 01:48:19.871084 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:19.871091 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:19.872605 (+ 1514us) server_negotiation.cc:407] Connection header received
0811 01:48:19.874833 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:19.874839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:19.874947 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:19.875064 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:19.876885 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.878034 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.879960 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.880395 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.880512 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:19.885352 (+ 4840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:19.885358 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:19.885361 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:19.885400 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:19.888755 (+ 3355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:19.889866 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:19.889875 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:19.889884 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:19.890034 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:19.890623 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:19.890627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:19.890630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:19.890955 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:19.891163 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:19.892089 (+ 926us) server_negotiation.cc:299] Negotiation successful
0811 01:48:19.892384 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":277,"threads_started":1}
I20260811 01:48:19.894074 8348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:19.871390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:43805)
0811 01:48:19.872395 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:19.872464 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 01:48:19.874485 (+ 2021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:48:19.875349 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:48:19.875358 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:48:19.875967 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:48:19.876669 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:19.876689 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.878273 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.878277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:19.879781 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:19.879797 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.880668 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.880677 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:19.880953 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:19.881896 (+ 943us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:48:19.881941 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:48:19.884972 (+ 3031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:48:19.888969 (+ 3997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:48:19.888991 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:48:19.889010 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:48:19.889675 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:48:19.890205 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:48:19.890210 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:48:19.890212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:48:19.890464 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:48:19.891115 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:48:19.891125 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:48:19.891569 (+ 444us) client_negotiation.cc:769] Sending connection context
0811 01:48:19.891951 (+ 382us) client_negotiation.cc:240] Negotiation successful
0811 01:48:19.892604 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":398,"threads_started":1}
I20260811 01:48:19.896510 8343 heartbeater.cc:347] Connected to a master server at 127.21.214.126:40013
I20260811 01:48:19.896915 8343 heartbeater.cc:464] Registering TS with master...
I20260811 01:48:19.897994 8343 heartbeater.cc:511] Master 127.21.214.126:40013 requested a full tablet report, sending...
I20260811 01:48:19.899739 7806 ts_manager.cc:194] Registered new tserver with Master: 704161483bfb45e2a571fa19ce99e7f4 (127.21.214.67:40965)
I20260811 01:48:19.900870 7806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:43805
I20260811 01:48:19.912204 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:48:19.933740 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:19.917947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38974 (local address 127.21.214.126:40013)
0811 01:48:19.918217 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:19.918227 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:19.918612 (+ 385us) server_negotiation.cc:407] Connection header received
0811 01:48:19.918768 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:19.918774 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:19.918885 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:19.919011 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:19.920624 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.921985 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.923619 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.924361 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.924506 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:19.927371 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:19.927376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:19.927380 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:19.927420 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:19.930707 (+ 3287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:19.931549 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:19.931554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:19.931556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:19.931700 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:19.932139 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:19.932142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:19.932145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:19.932558 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:19.932820 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:19.933268 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 01:48:19.933480 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 01:48:19.939086 7806 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:48:19.946591 7806 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:48:19.984320 8361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:19.962369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51672 (local address 127.21.214.67:40965)
0811 01:48:19.976625 (+ 14256us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:19.976636 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:19.976671 (+ 35us) server_negotiation.cc:407] Connection header received
0811 01:48:19.976799 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:19.976804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:19.976957 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:19.977200 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:48:19.977856 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.979498 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.981563 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.982974 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.983098 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:19.983357 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:19.983778 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 01:48:19.983945 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14076,"thread_start_us":13889,"threads_started":1}
I20260811 01:48:19.984383 8362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:19.961824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:40965 (local address 127.0.0.1:51672)
0811 01:48:19.972674 (+ 10850us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:19.972706 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 01:48:19.972940 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:48:19.977188 (+ 4248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:48:19.977193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:48:19.977219 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:48:19.977660 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:19.977670 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.979647 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.979652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:19.981405 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:19.981419 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.983327 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.983336 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:19.983510 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:19.983548 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 01:48:19.983695 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 01:48:19.983920 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10631,"mutex_wait_us":811,"thread_start_us":9612,"threads_started":1}
I20260811 01:48:19.985361 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:19.961206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:40303 (local address 127.0.0.1:52576)
0811 01:48:19.962155 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:19.962214 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 01:48:19.962429 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:48:19.975203 (+ 12774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:48:19.975216 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:48:19.975268 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:48:19.975959 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:19.975979 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.978825 (+ 2846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.978832 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:19.982068 (+ 3236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:19.982091 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.984313 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.984321 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:19.984553 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:19.984621 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 01:48:19.984711 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 01:48:19.984871 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"mutex_wait_us":73,"thread_start_us":403,"threads_started":1}
I20260811 01:48:19.985996 8364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:19.963286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:40993 (local address 127.0.0.1:45560)
0811 01:48:19.974369 (+ 11083us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:19.974388 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 01:48:19.974528 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:48:19.976092 (+ 1564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:48:19.976100 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:48:19.976132 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:48:19.976541 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:19.976564 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.978807 (+ 2243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.978812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:19.982091 (+ 3279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:19.982102 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.984678 (+ 2576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.984688 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:19.984891 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:19.984926 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 01:48:19.985030 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 01:48:19.985210 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10960,"mutex_wait_us":34,"thread_start_us":10784,"threads_started":1}
I20260811 01:48:19.986117 8363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:19.962905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45560 (local address 127.21.214.65:40993)
0811 01:48:19.975214 (+ 12309us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:19.975224 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:19.975256 (+ 32us) server_negotiation.cc:407] Connection header received
0811 01:48:19.975380 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:19.975388 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:19.975646 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:19.975885 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:48:19.976747 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.978594 (+ 1847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.982306 (+ 3712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.984041 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.984219 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:19.984484 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:19.985469 (+ 985us) server_negotiation.cc:299] Negotiation successful
0811 01:48:19.985736 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12113,"thread_start_us":12336,"threads_started":1}
I20260811 01:48:19.986047 8360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:19.961422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52576 (local address 127.21.214.66:40303)
0811 01:48:19.974020 (+ 12598us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:19.974036 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:19.974130 (+ 94us) server_negotiation.cc:407] Connection header received
0811 01:48:19.974302 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:19.974316 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:19.974748 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:19.976293 (+ 1545us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:48:19.976555 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.978664 (+ 2109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.982306 (+ 3642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:19.984041 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:19.985009 (+ 968us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:19.985263 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:19.985342 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 01:48:19.985710 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12322,"thread_start_us":12126,"threads_started":1}
I20260811 01:48:19.989486 8265 tablet_service.cc:1467] Processing CreateTablet for tablet 688a620907de432a982767e350d0cf5f (DEFAULT_TABLE table=test-table [id=95759ab1cf364e68baed389e69c89ba5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:48:19.990011 7950 tablet_service.cc:1467] Processing CreateTablet for tablet 688a620907de432a982767e350d0cf5f (DEFAULT_TABLE table=test-table [id=95759ab1cf364e68baed389e69c89ba5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:48:19.990154 8107 tablet_service.cc:1467] Processing CreateTablet for tablet 688a620907de432a982767e350d0cf5f (DEFAULT_TABLE table=test-table [id=95759ab1cf364e68baed389e69c89ba5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:48:19.991281 8265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 688a620907de432a982767e350d0cf5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:48:19.991464 7950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 688a620907de432a982767e350d0cf5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:48:19.991494 8107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 688a620907de432a982767e350d0cf5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:48:20.005400 8365 tablet_bootstrap.cc:492] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554: Bootstrap starting.
I20260811 01:48:20.010017 8367 tablet_bootstrap.cc:492] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2: Bootstrap starting.
I20260811 01:48:20.010303 8365 tablet_bootstrap.cc:654] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554: Neither blocks nor log segments found. Creating new log.
I20260811 01:48:20.011868 8365 log.cc:824] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554: Log is configured to *not* fsync() on all Append() calls
I20260811 01:48:20.012122 8368 tablet_bootstrap.cc:492] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4: Bootstrap starting.
I20260811 01:48:20.014154 8367 tablet_bootstrap.cc:654] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2: Neither blocks nor log segments found. Creating new log.
I20260811 01:48:20.015599 8367 log.cc:824] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2: Log is configured to *not* fsync() on all Append() calls
I20260811 01:48:20.015782 8368 tablet_bootstrap.cc:654] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4: Neither blocks nor log segments found. Creating new log.
I20260811 01:48:20.016275 8365 tablet_bootstrap.cc:492] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554: No bootstrap required, opened a new log
I20260811 01:48:20.016613 8365 ts_tablet_manager.cc:1397] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260811 01:48:20.017104 8368 log.cc:824] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4: Log is configured to *not* fsync() on all Append() calls
I20260811 01:48:20.020256 8367 tablet_bootstrap.cc:492] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2: No bootstrap required, opened a new log
I20260811 01:48:20.020514 8368 tablet_bootstrap.cc:492] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4: No bootstrap required, opened a new log
I20260811 01:48:20.020557 8367 ts_tablet_manager.cc:1397] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.002s
I20260811 01:48:20.020793 8368 ts_tablet_manager.cc:1397] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 01:48:20.028067 8365 raft_consensus.cc:348] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } } peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } }
I20260811 01:48:20.028142 8367 raft_consensus.cc:348] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } } peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } }
I20260811 01:48:20.028613 8365 raft_consensus.cc:374] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:48:20.028627 8367 raft_consensus.cc:374] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:48:20.028728 8365 raft_consensus.cc:729] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c86dbfd773a14e1181737ec35c0fc554, State: Initialized, Role: FOLLOWER
I20260811 01:48:20.028725 8367 raft_consensus.cc:729] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27c87199ae444777be34b8ce818f1ac2, State: Initialized, Role: FOLLOWER
I20260811 01:48:20.029422 8367 consensus_queue.cc:260] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } } peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } }
I20260811 01:48:20.029440 8365 consensus_queue.cc:260] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } } peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } }
I20260811 01:48:20.030670 8188 heartbeater.cc:503] Master 127.21.214.126:40013 was elected leader, sending a full tablet report...
I20260811 01:48:20.031445 8367 ts_tablet_manager.cc:1428] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2: Time spent starting tablet: real 0.011s user 0.009s sys 0.002s
I20260811 01:48:20.032438 8368 raft_consensus.cc:348] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } } peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } }
I20260811 01:48:20.032927 8368 raft_consensus.cc:374] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:48:20.033032 8368 raft_consensus.cc:729] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 704161483bfb45e2a571fa19ce99e7f4, State: Initialized, Role: FOLLOWER
I20260811 01:48:20.033627 8365 ts_tablet_manager.cc:1428] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
I20260811 01:48:20.033828 8368 consensus_queue.cc:260] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } } peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } }
I20260811 01:48:20.035090 8343 heartbeater.cc:503] Master 127.21.214.126:40013 was elected leader, sending a full tablet report...
I20260811 01:48:20.035804 8368 ts_tablet_manager.cc:1428] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
W20260811 01:48:20.119469 8344 tablet.cc:2369] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:48:20.154711 8025 tablet.cc:2369] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:48:20.261920 8189 tablet.cc:2369] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:48:20.404685 8374 raft_consensus.cc:482] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:48:20.405122 8374 raft_consensus.cc:504] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } } peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } }
I20260811 01:48:20.407269 8374 leader_election.cc:288] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27c87199ae444777be34b8ce818f1ac2 (127.21.214.66:40303), 704161483bfb45e2a571fa19ce99e7f4 (127.21.214.67:40965)
I20260811 01:48:20.415201 8360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:20.407893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:34223 (local address 127.21.214.66:40303)
0811 01:48:20.408104 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:20.408109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:20.408966 (+ 857us) server_negotiation.cc:407] Connection header received
0811 01:48:20.409260 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:20.409267 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:20.409346 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:20.409466 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:48:20.410060 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:20.410881 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:20.412997 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:20.414096 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:20.414288 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:20.414384 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:20.414722 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 01:48:20.414879 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 01:48:20.415448 8361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:20.407791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:41901 (local address 127.21.214.67:40965)
0811 01:48:20.408047 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:20.408058 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:20.409539 (+ 1481us) server_negotiation.cc:407] Connection header received
0811 01:48:20.409773 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:20.409779 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:20.409854 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:20.409966 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:48:20.410533 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:20.411297 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:20.413312 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:20.414314 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:20.414546 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:20.414639 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:20.414958 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 01:48:20.415114 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 01:48:20.415663 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:20.408246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:40965 (local address 127.21.214.65:41901)
0811 01:48:20.409409 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:20.409424 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 01:48:20.409610 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:48:20.409983 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:48:20.409988 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:48:20.410012 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:48:20.410360 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:20.410369 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:20.411492 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:20.411499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:20.413092 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:20.413104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:20.414570 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:20.414578 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:20.414784 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:20.414810 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 01:48:20.414927 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 01:48:20.415169 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1053,"mutex_wait_us":26,"thread_start_us":758,"threads_started":1}
I20260811 01:48:20.415948 8376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:20.407981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:40303 (local address 127.21.214.65:34223)
0811 01:48:20.408821 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:20.408840 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 01:48:20.409060 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:48:20.409488 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:48:20.409493 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:48:20.409522 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:48:20.409902 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:20.409911 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:20.411026 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:20.411034 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:20.412817 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:20.412834 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:20.414357 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:20.414364 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:20.414538 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:20.414568 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 01:48:20.414693 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 01:48:20.414879 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":30,"thread_start_us":505,"threads_started":1}
I20260811 01:48:20.416888 8291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "688a620907de432a982767e350d0cf5f" candidate_uuid: "c86dbfd773a14e1181737ec35c0fc554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "704161483bfb45e2a571fa19ce99e7f4" is_pre_election: true
I20260811 01:48:20.417410 8291 raft_consensus.cc:2454] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c86dbfd773a14e1181737ec35c0fc554 in term 0.
I20260811 01:48:20.417270 8130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "688a620907de432a982767e350d0cf5f" candidate_uuid: "c86dbfd773a14e1181737ec35c0fc554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27c87199ae444777be34b8ce818f1ac2" is_pre_election: true
I20260811 01:48:20.417862 8130 raft_consensus.cc:2454] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c86dbfd773a14e1181737ec35c0fc554 in term 0.
I20260811 01:48:20.418057 7898 leader_election.cc:302] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 704161483bfb45e2a571fa19ce99e7f4, c86dbfd773a14e1181737ec35c0fc554; no voters:
I20260811 01:48:20.418632 8374 raft_consensus.cc:2781] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:48:20.418835 8374 raft_consensus.cc:482] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:48:20.418951 8374 raft_consensus.cc:3037] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:48:20.422075 8374 raft_consensus.cc:504] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } } peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } }
I20260811 01:48:20.422845 8374 leader_election.cc:288] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [CANDIDATE]: Term 1 election: Requested vote from peers 27c87199ae444777be34b8ce818f1ac2 (127.21.214.66:40303), 704161483bfb45e2a571fa19ce99e7f4 (127.21.214.67:40965)
I20260811 01:48:20.423492 8130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "688a620907de432a982767e350d0cf5f" candidate_uuid: "c86dbfd773a14e1181737ec35c0fc554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27c87199ae444777be34b8ce818f1ac2"
I20260811 01:48:20.423746 8130 raft_consensus.cc:3037] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:48:20.423712 8291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "688a620907de432a982767e350d0cf5f" candidate_uuid: "c86dbfd773a14e1181737ec35c0fc554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "704161483bfb45e2a571fa19ce99e7f4"
I20260811 01:48:20.424005 8291 raft_consensus.cc:3037] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:48:20.426311 8130 raft_consensus.cc:2454] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c86dbfd773a14e1181737ec35c0fc554 in term 1.
I20260811 01:48:20.426620 8291 raft_consensus.cc:2454] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c86dbfd773a14e1181737ec35c0fc554 in term 1.
I20260811 01:48:20.426878 7897 leader_election.cc:302] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27c87199ae444777be34b8ce818f1ac2, c86dbfd773a14e1181737ec35c0fc554; no voters:
I20260811 01:48:20.427234 8374 raft_consensus.cc:2781] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:48:20.427728 8374 raft_consensus.cc:686] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [term 1 LEADER]: Becoming Leader. State: Replica: c86dbfd773a14e1181737ec35c0fc554, State: Running, Role: LEADER
I20260811 01:48:20.428344 8374 consensus_queue.cc:237] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } } peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } }
I20260811 01:48:20.433259 7804 catalog_manager.cc:5495] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 reported cstate change: term changed from 0 to 1, leader changed from <none> to c86dbfd773a14e1181737ec35c0fc554 (127.21.214.65). New cstate: current_term: 1 leader_uuid: "c86dbfd773a14e1181737ec35c0fc554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } health_report { overall_health: UNKNOWN } } }
I20260811 01:48:20.560658 8383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:20.553194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45562 (local address 127.21.214.65:40993)
0811 01:48:20.554113 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:20.554120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:20.554142 (+ 22us) server_negotiation.cc:407] Connection header received
0811 01:48:20.554241 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:20.554245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:20.554344 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:20.554537 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:48:20.555273 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:20.556187 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:20.558139 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:20.558584 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:20.558705 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:20.559259 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:48:20.559916 (+ 657us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:48:20.560039 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:20.560207 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 01:48:20.560292 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":624,"threads_started":1}
I20260811 01:48:20.570257 8130 raft_consensus.cc:1261] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2 [term 1 FOLLOWER]: Refusing update from remote peer c86dbfd773a14e1181737ec35c0fc554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:48:20.570258 8291 raft_consensus.cc:1261] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4 [term 1 FOLLOWER]: Refusing update from remote peer c86dbfd773a14e1181737ec35c0fc554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:48:20.571641 8379 consensus_queue.cc:1035] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:48:20.572201 8374 consensus_queue.cc:1035] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:48:20.586978 8384 mvcc.cc:205] Tried to move back new op lower bound from 7317147240723550208 to 7317147240158871552. Current Snapshot: MvccSnapshot[applied={T|T < 7317147240723550208}]
I20260811 01:48:20.597569 8387 mvcc.cc:205] Tried to move back new op lower bound from 7317147240723550208 to 7317147240158871552. Current Snapshot: MvccSnapshot[applied={T|T < 7317147240723550208}]
I20260811 01:48:20.605556 7804 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:48:20.605901 7804 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:48:20.610569 7804 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: Sending DeleteTablet for 3 replicas of tablet 688a620907de432a982767e350d0cf5f
I20260811 01:48:20.612335 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 7879
I20260811 01:48:20.612342 8107 tablet_service.cc:1514] Processing DeleteTablet for tablet 688a620907de432a982767e350d0cf5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:48:20 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:52576
I20260811 01:48:20.612583 7950 tablet_service.cc:1514] Processing DeleteTablet for tablet 688a620907de432a982767e350d0cf5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:48:20 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:45560
I20260811 01:48:20.613206 8265 tablet_service.cc:1514] Processing DeleteTablet for tablet 688a620907de432a982767e350d0cf5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:48:20 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:51672
I20260811 01:48:20.621811 8397 tablet_replica.cc:330] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554: stopping tablet replica
I20260811 01:48:20.622434 8397 raft_consensus.cc:2229] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:48:20.623335 8397 raft_consensus.cc:2258] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:48:20.623699 8398 tablet_replica.cc:330] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4: stopping tablet replica
I20260811 01:48:20.624337 8398 raft_consensus.cc:2229] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:48:20.625087 8398 raft_consensus.cc:2258] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:48:20.626382 8396 tablet_replica.cc:330] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2: stopping tablet replica
I20260811 01:48:20.626931 8396 raft_consensus.cc:2229] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:48:20.627544 8396 raft_consensus.cc:2258] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:48:20.627810 8398 ts_tablet_manager.cc:1905] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:48:20.629796 8396 ts_tablet_manager.cc:1905] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 01:48:20 dist-test-slave-9gcw krb5kdc[7756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:48:20.636345 8398 ts_tablet_manager.cc:1918] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:48:20.636646 8398 log.cc:1196] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/688a620907de432a982767e350d0cf5f
I20260811 01:48:20.637336 8398 ts_tablet_manager.cc:1939] T 688a620907de432a982767e350d0cf5f P 704161483bfb45e2a571fa19ce99e7f4: Deleting consensus metadata
I20260811 01:48:20.637876 8396 ts_tablet_manager.cc:1918] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:48:20.638108 8396 log.cc:1196] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/688a620907de432a982767e350d0cf5f
I20260811 01:48:20.638736 8396 ts_tablet_manager.cc:1939] T 688a620907de432a982767e350d0cf5f P 27c87199ae444777be34b8ce818f1ac2: Deleting consensus metadata
I20260811 01:48:20.640590 7790 catalog_manager.cc:4841] TS 704161483bfb45e2a571fa19ce99e7f4 (127.21.214.67:40965): tablet 688a620907de432a982767e350d0cf5f (table test-table [id=95759ab1cf364e68baed389e69c89ba5]) successfully deleted
I20260811 01:48:20.641075 7789 catalog_manager.cc:4841] TS 27c87199ae444777be34b8ce818f1ac2 (127.21.214.66:40303): tablet 688a620907de432a982767e350d0cf5f (table test-table [id=95759ab1cf364e68baed389e69c89ba5]) successfully deleted
I20260811 01:48:20.642426 8397 ts_tablet_manager.cc:1905] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:48:20.647578 8383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:20.614555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45568 (local address 127.21.214.65:40993)
0811 01:48:20.614743 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:20.614749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:20.622688 (+ 7939us) server_negotiation.cc:407] Connection header received
0811 01:48:20.623060 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:20.623066 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:20.623199 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:20.623339 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:20.625834 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:20.627117 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:20.629458 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:20.630200 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:20.630327 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:20.637642 (+ 7315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:20.637660 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:20.637680 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:20.637745 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:20.642990 (+ 5245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:20.644299 (+ 1309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:20.644313 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:20.644317 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:20.644530 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:20.645081 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:20.645087 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:20.645090 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:20.645683 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:20.646053 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:20.646406 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 01:48:20.646764 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 01:48:20.648540 8019 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:48:20.757175 8397 ts_tablet_manager.cc:1918] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:48:20.757441 8397 log.cc:1196] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/688a620907de432a982767e350d0cf5f
I20260811 01:48:20.758078 8397 ts_tablet_manager.cc:1939] T 688a620907de432a982767e350d0cf5f P c86dbfd773a14e1181737ec35c0fc554: Deleting consensus metadata
I20260811 01:48:20.759838 7789 catalog_manager.cc:4841] TS c86dbfd773a14e1181737ec35c0fc554 (127.21.214.65:40993): tablet 688a620907de432a982767e350d0cf5f (table test-table [id=95759ab1cf364e68baed389e69c89ba5]) successfully deleted
I20260811 01:48:20.760380 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 7879
I20260811 01:48:20.787513 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 8030
Aug 11 01:48:20 dist-test-slave-9gcw krb5kdc[7756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.66@KRBTEST.COM
I20260811 01:48:20.805563 8360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:20.788394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52586 (local address 127.21.214.66:40303)
0811 01:48:20.788576 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:20.788581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:20.788600 (+ 19us) server_negotiation.cc:407] Connection header received
0811 01:48:20.788777 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:20.788781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:20.788848 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:20.788938 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:20.790753 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:20.791785 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:20.793338 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:20.793779 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:20.793889 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:20.798969 (+ 5080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:20.798987 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:20.799005 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:20.799059 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:20.802645 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:20.803445 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:20.803453 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:20.803455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:20.803594 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:20.803981 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:20.803985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:20.803987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:20.804408 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:20.804667 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:20.805035 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 01:48:20.805286 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 01:48:20.806345 8183 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:48:20.910275 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 8030
I20260811 01:48:20.941397 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 8195
Aug 11 01:48:20 dist-test-slave-9gcw krb5kdc[7756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.67@KRBTEST.COM
I20260811 01:48:20.959612 8402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:20.942968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51680 (local address 127.21.214.67:40965)
0811 01:48:20.943703 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:20.943711 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:20.943736 (+ 25us) server_negotiation.cc:407] Connection header received
0811 01:48:20.943808 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:20.943813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:20.943921 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:20.944063 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:20.945919 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:20.946682 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:20.947941 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:20.948391 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:20.948491 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:20.953500 (+ 5009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:20.953517 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:20.953536 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:20.953580 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:20.956529 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:20.957436 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:20.957444 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:20.957446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:20.957599 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:20.958153 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:20.958156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:20.958158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:20.958572 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:20.958805 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:20.959138 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 01:48:20.959341 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":423,"threads_started":1}
I20260811 01:48:20.960302 8338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:48:21.053805 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 8195
I20260811 01:48:21.076500 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 7771
I20260811 01:48:21.077962 7843 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:48:21.193487 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 7771
I20260811 01:48:41.225042 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:40013
--webserver_interface=127.21.214.126
--webserver_port=35031
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:40013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:48:41.601478 8406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:48:41.601835 8406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:48:41.601930 8406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:48:41.609622 8406 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 01:48:41.609745 8406 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 01:48:41.609786 8406 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 01:48:41.609853 8406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:48:41.609911 8406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:48:41.609985 8406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:48:41.610034 8406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:48:41.610095 8406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:48:41.621356 8406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:40013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:40013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=35031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:48:41.622869 8406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:48:41.624353 8406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:48:41.654618 8414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:48:41.654597 8416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:48:41.654654 8413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:48:41.655603 8406 server_base.cc:1034] running on GCE node
I20260811 01:48:41.656711 8406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:48:41.658776 8406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:48:41.660084 8406 hybrid_clock.cc:648] HybridClock initialized: now 1786412921660000 us; error 59 us; skew 500 ppm
I20260811 01:48:41.660579 8406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:48:41 dist-test-slave-9gcw krb5kdc[7756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412921, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:48:41.669821 8406 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:48:41.671844 8406 webserver.cc:460] Webserver started at http://127.21.214.126:35031/ using document root <none> and password file <none>
I20260811 01:48:41.672750 8406 fs_manager.cc:359] Metadata directory not provided
I20260811 01:48:41.672883 8406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:48:41.680400 8406 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 01:48:41.684623 8424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:41.686504 8406 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 01:48:41.686730 8406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "fc0a298e73dd4848ae307d38c888ded9"
format_stamp: "Formatted at 2026-08-11 01:48:17 on dist-test-slave-9gcw"
server_key: "10087ef2ea7922aab8b3efddba6985ef"
server_key_iv: "317d1cde71154733a98094111cb40be4"
server_key_version: "encryptionkey@0"
I20260811 01:48:41.688058 8406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:48:41.726781 8406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:48:41.732522 8406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:48:41.732928 8406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:48:41.756610 8406 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:40013
I20260811 01:48:41.756618 8482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:40013 every 8 connection(s)
I20260811 01:48:41.759258 8406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:48:41.763193 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 8406
I20260811 01:48:41.764256 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:40993
--local_ip_for_outbound_sockets=127.21.214.65
--tserver_master_addrs=127.21.214.126:40013
--webserver_port=46845
--webserver_interface=127.21.214.65
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:48:41.767988 8483 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:48:41.771941 8483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: Bootstrap starting.
I20260811 01:48:41.794685 8483 log.cc:824] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: Log is configured to *not* fsync() on all Append() calls
I20260811 01:48:41.808022 8483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:48:41.808689 8483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: Bootstrap complete.
I20260811 01:48:41.819083 8483 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0a298e73dd4848ae307d38c888ded9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40013 } }
I20260811 01:48:41.820037 8483 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0a298e73dd4848ae307d38c888ded9, State: Initialized, Role: FOLLOWER
I20260811 01:48:41.820778 8483 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0a298e73dd4848ae307d38c888ded9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40013 } }
I20260811 01:48:41.821060 8483 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:48:41.821233 8483 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:48:41.821437 8483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 1 FOLLOWER]: Advancing to term 2
I20260811 01:48:41.825775 8483 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0a298e73dd4848ae307d38c888ded9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40013 } }
I20260811 01:48:41.826385 8483 leader_election.cc:302] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc0a298e73dd4848ae307d38c888ded9; no voters:
I20260811 01:48:41.827102 8483 leader_election.cc:288] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 01:48:41.827297 8488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 01:48:41.828814 8488 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 2 LEADER]: Becoming Leader. State: Replica: fc0a298e73dd4848ae307d38c888ded9, State: Running, Role: LEADER
I20260811 01:48:41.829487 8488 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0a298e73dd4848ae307d38c888ded9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40013 } }
I20260811 01:48:41.829932 8483 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:48:41.832643 8490 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc0a298e73dd4848ae307d38c888ded9. Latest consensus state: current_term: 2 leader_uuid: "fc0a298e73dd4848ae307d38c888ded9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0a298e73dd4848ae307d38c888ded9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40013 } } }
I20260811 01:48:41.833181 8490 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [sys.catalog]: This master's current role is: LEADER
I20260811 01:48:41.833778 8489 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fc0a298e73dd4848ae307d38c888ded9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0a298e73dd4848ae307d38c888ded9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40013 } } }
I20260811 01:48:41.834177 8489 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [sys.catalog]: This master's current role is: LEADER
I20260811 01:48:41.844728 8496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:48:41.854300 8496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:48:41.856109 8496 catalog_manager.cc:1259] Loaded cluster ID: d7f0f9cd8c9a43daa9374df302479ef7
I20260811 01:48:41.856277 8496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:48:41.861042 8496 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:48:41.862972 8496 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: Loaded TSK: 0
I20260811 01:48:41.923512 8496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: Generated new TSK 1
I20260811 01:48:41.924008 8496 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 01:48:42.174604 8486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:48:42.174995 8486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:48:42.175194 8486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:48:42.182994 8486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:48:42.183140 8486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:48:42.183339 8486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:48:42.194802 8486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:40993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=46845
--tserver_master_addrs=127.21.214.126:40013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:48:42.196291 8486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:48:42.197829 8486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:48:42.212137 8515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:48:42.212150 8516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:48:42.212755 8518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:48:42.214250 8486 server_base.cc:1034] running on GCE node
I20260811 01:48:42.215121 8486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:48:42.216225 8486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:48:42.217454 8486 hybrid_clock.cc:648] HybridClock initialized: now 1786412922217402 us; error 37 us; skew 500 ppm
I20260811 01:48:42.217890 8486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:48:42 dist-test-slave-9gcw krb5kdc[7756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412922, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:48:42.221698 8486 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:48:42.223580 8486 webserver.cc:460] Webserver started at http://127.21.214.65:46845/ using document root <none> and password file <none>
I20260811 01:48:42.224596 8486 fs_manager.cc:359] Metadata directory not provided
I20260811 01:48:42.224774 8486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:48:42.233013 8486 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 01:48:42.239009 8525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:42.241731 8486 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.005s sys 0.000s
I20260811 01:48:42.242082 8486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "c86dbfd773a14e1181737ec35c0fc554"
format_stamp: "Formatted at 2026-08-11 01:48:18 on dist-test-slave-9gcw"
server_key: "25260aa86b4b07b8d9c35cd5bf02c4c9"
server_key_iv: "954e7a3c2d74fb2d465c9a0079c9aa28"
server_key_version: "encryptionkey@0"
I20260811 01:48:42.244056 8486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:48:42.291402 8486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:48:42.297361 8486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:48:42.297796 8486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:48:42.298957 8486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:48:42.300475 8486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:48:42.300623 8486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:42.300761 8486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:48:42.300856 8486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:42.345310 8486 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:40993
I20260811 01:48:42.345404 8661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:40993 every 8 connection(s)
I20260811 01:48:42.347208 8486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:48:42.349222 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 8486
I20260811 01:48:42.350283 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:40303
--local_ip_for_outbound_sockets=127.21.214.66
--tserver_master_addrs=127.21.214.126:40013
--webserver_port=36197
--webserver_interface=127.21.214.66
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 01:48:42 dist-test-slave-9gcw krb5kdc[7756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412922, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:48:42.393303 8666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:42.350846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:54135 (local address 127.21.214.126:40013)
0811 01:48:42.363915 (+ 13069us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:42.363937 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:42.363994 (+ 57us) server_negotiation.cc:407] Connection header received
0811 01:48:42.364634 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:42.364664 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:42.365296 (+ 632us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:42.368156 (+ 2860us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:42.368678 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:42.370892 (+ 2214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:42.373770 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:42.375153 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:42.375407 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:42.381504 (+ 6097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:42.381520 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:42.381551 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:42.381616 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:42.386728 (+ 5112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:42.388114 (+ 1386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:42.388129 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:42.388142 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:42.388440 (+ 298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:42.389089 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:42.389095 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:42.389098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:42.389980 (+ 882us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:42.390196 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:42.391398 (+ 1202us) server_negotiation.cc:299] Negotiation successful
0811 01:48:42.391855 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12786,"thread_start_us":12658,"threads_started":1}
I20260811 01:48:42.394299 8668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:42.351239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:54135)
0811 01:48:42.357913 (+ 6674us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:42.357996 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 01:48:42.360994 (+ 2998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:48:42.365981 (+ 4987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:48:42.365997 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:48:42.366883 (+ 886us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:48:42.367785 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:42.367838 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:42.371113 (+ 3275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:42.371121 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:42.373573 (+ 2452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:42.373595 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:42.375473 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:42.375483 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:42.375876 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:42.376776 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:48:42.376814 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:48:42.380721 (+ 3907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:48:42.386984 (+ 6263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:48:42.387002 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:48:42.387028 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:48:42.387885 (+ 857us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:48:42.388617 (+ 732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:48:42.388625 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:48:42.388633 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:48:42.388931 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:48:42.390176 (+ 1245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:48:42.390189 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:48:42.390725 (+ 536us) client_negotiation.cc:769] Sending connection context
0811 01:48:42.391103 (+ 378us) client_negotiation.cc:240] Negotiation successful
0811 01:48:42.391825 (+ 722us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6251,"thread_start_us":5957,"threads_started":1}
I20260811 01:48:42.398269 8664 heartbeater.cc:347] Connected to a master server at 127.21.214.126:40013
I20260811 01:48:42.398656 8664 heartbeater.cc:464] Registering TS with master...
I20260811 01:48:42.399552 8664 heartbeater.cc:511] Master 127.21.214.126:40013 requested a full tablet report, sending...
I20260811 01:48:42.402594 8444 ts_manager.cc:194] Registered new tserver with Master: c86dbfd773a14e1181737ec35c0fc554 (127.21.214.65:40993)
I20260811 01:48:42.405182 8444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:54135
W20260811 01:48:42.724385 8669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:48:42.724749 8669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:48:42.724938 8669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:48:42.732930 8669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:48:42.733091 8669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:48:42.733349 8669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:48:42.743525 8669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:40303
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=36197
--tserver_master_addrs=127.21.214.126:40013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:48:42.744978 8669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:48:42.746552 8669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:48:42.761526 8674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:48:42.761618 8675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:48:42.762285 8679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:48:42.762591 8669 server_base.cc:1034] running on GCE node
I20260811 01:48:42.763373 8669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:48:42.764612 8669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:48:42.765851 8669 hybrid_clock.cc:648] HybridClock initialized: now 1786412922765805 us; error 27 us; skew 500 ppm
I20260811 01:48:42.766225 8669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:48:42 dist-test-slave-9gcw krb5kdc[7756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412922, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:48:42.770303 8669 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:48:42.774456 8669 webserver.cc:460] Webserver started at http://127.21.214.66:36197/ using document root <none> and password file <none>
I20260811 01:48:42.775264 8669 fs_manager.cc:359] Metadata directory not provided
I20260811 01:48:42.775475 8669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:48:42.781674 8669 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 01:48:42.784842 8687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:42.786509 8669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:48:42.786757 8669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "27c87199ae444777be34b8ce818f1ac2"
format_stamp: "Formatted at 2026-08-11 01:48:19 on dist-test-slave-9gcw"
server_key: "44df448c0f3782a0c495cf278822c28a"
server_key_iv: "252f47ffff186787770c7efc623d64af"
server_key_version: "encryptionkey@0"
I20260811 01:48:42.787911 8669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:48:42.812474 8669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:48:42.818099 8669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:48:42.818457 8669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:48:42.819595 8669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:48:42.821206 8669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:48:42.821348 8669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 01:48:42.821501 8669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:48:42.821600 8669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:42.859242 8669 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:40303
I20260811 01:48:42.859346 8820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:40303 every 8 connection(s)
I20260811 01:48:42.860966 8669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:48:42.870290 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 8669
I20260811 01:48:42.871052 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:40965
--local_ip_for_outbound_sockets=127.21.214.67
--tserver_master_addrs=127.21.214.126:40013
--webserver_port=39387
--webserver_interface=127.21.214.67
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 01:48:42 dist-test-slave-9gcw krb5kdc[7756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412922, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:48:42.883121 8666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:42.863899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:58873 (local address 127.21.214.126:40013)
0811 01:48:42.864114 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:42.864122 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:42.865567 (+ 1445us) server_negotiation.cc:407] Connection header received
0811 01:48:42.867847 (+ 2280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:42.867852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:42.867934 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:42.868028 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:42.869772 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:42.870581 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:42.872334 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:42.872775 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:42.872917 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:42.876953 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:42.876958 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:42.876961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:42.877000 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:42.879800 (+ 2800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:42.880595 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:42.880599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:42.880601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:42.880734 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:42.881241 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:42.881246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:42.881248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:42.881562 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:42.881731 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:42.882477 (+ 746us) server_negotiation.cc:299] Negotiation successful
0811 01:48:42.882730 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 01:48:42.884353 8823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:42.864393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:58873)
0811 01:48:42.865371 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:42.865436 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 01:48:42.867548 (+ 2112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:48:42.868299 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:48:42.868308 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:48:42.868863 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:48:42.869543 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:42.869562 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:42.870749 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:42.870754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:42.872172 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:42.872185 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:42.873039 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:42.873047 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:42.873340 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:42.874121 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:48:42.874148 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:48:42.876700 (+ 2552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:48:42.879949 (+ 3249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:48:42.879959 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:48:42.879977 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:48:42.880445 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:48:42.880867 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:48:42.880872 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:48:42.880876 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:48:42.881081 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:48:42.881701 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:48:42.881709 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:48:42.882073 (+ 364us) client_negotiation.cc:769] Sending connection context
0811 01:48:42.882374 (+ 301us) client_negotiation.cc:240] Negotiation successful
0811 01:48:42.882996 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":363,"threads_started":1}
I20260811 01:48:42.888819 8821 heartbeater.cc:347] Connected to a master server at 127.21.214.126:40013
I20260811 01:48:42.889241 8821 heartbeater.cc:464] Registering TS with master...
I20260811 01:48:42.890384 8821 heartbeater.cc:511] Master 127.21.214.126:40013 requested a full tablet report, sending...
I20260811 01:48:42.894537 8444 ts_manager.cc:194] Registered new tserver with Master: 27c87199ae444777be34b8ce818f1ac2 (127.21.214.66:40303)
I20260811 01:48:42.895788 8444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:58873
W20260811 01:48:43.257951 8824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:48:43.258304 8824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:48:43.258416 8824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:48:43.266330 8824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:48:43.266494 8824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:48:43.266625 8824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:48:43.277557 8824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:40965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=39387
--tserver_master_addrs=127.21.214.126:40013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:48:43.279002 8824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:48:43.280424 8824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:48:43.295607 8831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:48:43.295714 8834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:48:43.295718 8832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:48:43.296891 8824 server_base.cc:1034] running on GCE node
I20260811 01:48:43.298012 8824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:48:43.299115 8824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:48:43.300343 8824 hybrid_clock.cc:648] HybridClock initialized: now 1786412923300289 us; error 37 us; skew 500 ppm
I20260811 01:48:43.300781 8824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:48:43 dist-test-slave-9gcw krb5kdc[7756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412923, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:48:43.304239 8824 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:48:43.306051 8824 webserver.cc:460] Webserver started at http://127.21.214.67:39387/ using document root <none> and password file <none>
I20260811 01:48:43.306813 8824 fs_manager.cc:359] Metadata directory not provided
I20260811 01:48:43.306988 8824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:48:43.313599 8824 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 01:48:43.316577 8841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:43.318024 8824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:48:43.318219 8824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "704161483bfb45e2a571fa19ce99e7f4"
format_stamp: "Formatted at 2026-08-11 01:48:19 on dist-test-slave-9gcw"
server_key: "82a8861f769c6a7f85d1b337619c2497"
server_key_iv: "a1097fee717e69b6e422853bf81a3181"
server_key_version: "encryptionkey@0"
I20260811 01:48:43.319268 8824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:48:43.343904 8824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:48:43.349733 8824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:48:43.350035 8824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:48:43.351270 8824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:48:43.352792 8824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:48:43.352895 8824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:43.352974 8824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:48:43.353031 8824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:48:43.388542 8824 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:40965
I20260811 01:48:43.388653 8971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:40965 every 8 connection(s)
I20260811 01:48:43.390107 8824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:48:43.392094 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 8824
I20260811 01:48:43.409425 8664 heartbeater.cc:503] Master 127.21.214.126:40013 was elected leader, sending a full tablet report...
Aug 11 01:48:43 dist-test-slave-9gcw krb5kdc[7756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412923, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:48:43.426529 8975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:43.393163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51458 (local address 127.21.214.126:40013)
0811 01:48:43.408122 (+ 14959us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:43.408126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:43.408143 (+ 17us) server_negotiation.cc:407] Connection header received
0811 01:48:43.408197 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:43.408201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:43.408261 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:43.408362 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:43.411195 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.412394 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.414479 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.415149 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.415274 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:43.419160 (+ 3886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:43.419166 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:43.419169 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:43.419201 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:43.423062 (+ 3861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:43.424125 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:43.424132 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:43.424139 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:43.424314 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:43.424746 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:43.424751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:43.424755 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:43.425188 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:43.425447 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:43.425917 (+ 470us) server_negotiation.cc:299] Negotiation successful
0811 01:48:43.426171 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14847,"thread_start_us":12519,"threads_started":1}
I20260811 01:48:43.428560 8976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:43.393091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:59637 (local address 127.21.214.126:40013)
0811 01:48:43.405652 (+ 12561us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:43.405663 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:43.407075 (+ 1412us) server_negotiation.cc:407] Connection header received
0811 01:48:43.409879 (+ 2804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:43.409883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:43.409941 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:43.410076 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:43.412504 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.413786 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.415952 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.416706 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.416871 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:43.421109 (+ 4238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:43.421118 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:43.421123 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:43.421217 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:43.424775 (+ 3558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:43.425842 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:43.425852 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:43.425855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:43.425980 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:43.426583 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:43.426587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:43.426590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:43.426976 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:43.427177 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:43.427933 (+ 756us) server_negotiation.cc:299] Negotiation successful
0811 01:48:43.428165 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12345,"thread_start_us":14751,"threads_started":1}
I20260811 01:48:43.429713 8979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:43.393885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:59637)
0811 01:48:43.406893 (+ 13008us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:43.406943 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 01:48:43.409496 (+ 2553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:48:43.410456 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:48:43.410472 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:48:43.411380 (+ 908us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:48:43.412250 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:43.412277 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.413985 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.413989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:43.415778 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:43.415791 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.416959 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.416965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:43.417266 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:43.417999 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:48:43.418022 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:48:43.420723 (+ 2701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:48:43.424927 (+ 4204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:48:43.424935 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:48:43.424950 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:48:43.425720 (+ 770us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:48:43.426120 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:48:43.426125 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:48:43.426129 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:48:43.426437 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:48:43.427134 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:48:43.427143 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:48:43.427543 (+ 400us) client_negotiation.cc:769] Sending connection context
0811 01:48:43.427841 (+ 298us) client_negotiation.cc:240] Negotiation successful
0811 01:48:43.428435 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12657,"thread_start_us":12387,"threads_started":1}
I20260811 01:48:43.431130 8972 heartbeater.cc:347] Connected to a master server at 127.21.214.126:40013
I20260811 01:48:43.431542 8972 heartbeater.cc:464] Registering TS with master...
I20260811 01:48:43.432287 8972 heartbeater.cc:511] Master 127.21.214.126:40013 requested a full tablet report, sending...
I20260811 01:48:43.433869 8444 ts_manager.cc:194] Registered new tserver with Master: 704161483bfb45e2a571fa19ce99e7f4 (127.21.214.67:40965)
I20260811 01:48:43.434731 8444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:59637
I20260811 01:48:43.442250 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:48:43.449534 8976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:43.443241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51474 (local address 127.21.214.126:40013)
0811 01:48:43.443471 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:43.443480 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:43.443911 (+ 431us) server_negotiation.cc:407] Connection header received
0811 01:48:43.444088 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:43.444092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:43.444258 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:43.444402 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:48:43.444963 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.445786 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.447408 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.447828 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.447924 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:43.448460 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:48:43.448818 (+ 358us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 01:48:43.449208 (+ 390us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51474: token expired
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 01:48:43.449692 8976 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51474: token expired
W20260811 01:48:43.450336 8352 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.21.214.126:40013)
I20260811 01:48:43.465788 8976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:43.450941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51476 (local address 127.21.214.126:40013)
0811 01:48:43.451114 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:43.451118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:43.451135 (+ 17us) server_negotiation.cc:407] Connection header received
0811 01:48:43.451279 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:43.451283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:43.451344 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:43.451430 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:43.453260 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.454124 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.455806 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.456292 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.456401 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:43.459676 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:43.459682 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:43.459685 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:43.459724 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:43.462802 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:43.463685 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:43.463691 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:43.463693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:43.463865 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:43.464291 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:43.464296 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:43.464298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:43.464669 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:43.464929 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:43.465307 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 01:48:43.465529 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 01:48:43.469882 8444 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:48:43.478097 8444 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:48:43.502264 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:43.491761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50926 (local address 127.21.214.65:40993)
0811 01:48:43.492598 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:43.492611 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:43.493582 (+ 971us) server_negotiation.cc:407] Connection header received
0811 01:48:43.493715 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:43.493723 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:43.494008 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:43.494243 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:48:43.495077 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.496519 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.499514 (+ 2995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.500751 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.500953 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:43.501311 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:43.501733 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 01:48:43.501934 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":393,"threads_started":1}
I20260811 01:48:43.502291 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:43.491399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:40993 (local address 127.0.0.1:50926)
0811 01:48:43.492936 (+ 1537us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:43.492974 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 01:48:43.493369 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:48:43.494260 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:48:43.494267 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:48:43.494305 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:48:43.494893 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:43.494908 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.496723 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.496728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:43.499269 (+ 2541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:43.499286 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.501035 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.501042 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:43.501421 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:43.501467 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 01:48:43.501639 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 01:48:43.501859 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1299,"mutex_wait_us":69,"thread_start_us":2177,"threads_started":1}
I20260811 01:48:43.503441 8987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:43.491982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:40965 (local address 127.0.0.1:57478)
0811 01:48:43.493843 (+ 1861us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:43.493861 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:48:43.493983 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:48:43.494965 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:48:43.494969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:48:43.494996 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:48:43.495395 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:43.495413 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.497572 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.497577 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:43.499807 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:43.499837 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.502597 (+ 2760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.502602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:43.502734 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:43.502758 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 01:48:43.502824 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 01:48:43.502994 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1728,"thread_start_us":576,"threads_started":1}
I20260811 01:48:43.504347 8990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:43.492846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57478 (local address 127.21.214.67:40965)
0811 01:48:43.493631 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:43.493640 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:43.493972 (+ 332us) server_negotiation.cc:407] Connection header received
0811 01:48:43.494158 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:43.494168 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:43.494421 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:43.494613 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:48:43.495677 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.497402 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.500054 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.501709 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.501841 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:43.503636 (+ 1795us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:43.503747 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 01:48:43.503949 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":443,"threads_started":1}
I20260811 01:48:43.505662 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:43.494076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:40303 (local address 127.0.0.1:58564)
0811 01:48:43.494953 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:43.494972 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 01:48:43.496048 (+ 1076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:48:43.496958 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:48:43.496963 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:48:43.496988 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:48:43.497452 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:43.497469 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.499117 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.499121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:43.501429 (+ 2308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:43.501445 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.504845 (+ 3400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.504850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:43.504975 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:43.504997 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 01:48:43.505109 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 01:48:43.505266 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":87,"thread_start_us":734,"threads_started":1}
I20260811 01:48:43.505707 8993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:43.494572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58564 (local address 127.21.214.66:40303)
0811 01:48:43.495471 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:43.495479 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:43.495509 (+ 30us) server_negotiation.cc:407] Connection header received
0811 01:48:43.496266 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:43.496274 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:43.496495 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:43.496686 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:48:43.497668 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.498948 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.501617 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.504357 (+ 2740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.504531 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:43.504713 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:43.505185 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 01:48:43.505373 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":570,"threads_started":1}
I20260811 01:48:43.507032 8585 tablet_service.cc:1467] Processing CreateTablet for tablet 64e0e2654a8a446089289affa209c21e (DEFAULT_TABLE table=test-table [id=e2c89c87de7a4e6aa4547c569a8cc725]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:48:43.508409 8585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64e0e2654a8a446089289affa209c21e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:48:43.509063 8901 tablet_service.cc:1467] Processing CreateTablet for tablet 64e0e2654a8a446089289affa209c21e (DEFAULT_TABLE table=test-table [id=e2c89c87de7a4e6aa4547c569a8cc725]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:48:43.509450 8744 tablet_service.cc:1467] Processing CreateTablet for tablet 64e0e2654a8a446089289affa209c21e (DEFAULT_TABLE table=test-table [id=e2c89c87de7a4e6aa4547c569a8cc725]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:48:43.510686 8744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64e0e2654a8a446089289affa209c21e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:48:43.510689 8901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64e0e2654a8a446089289affa209c21e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:48:43.522822 8994 tablet_bootstrap.cc:492] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554: Bootstrap starting.
I20260811 01:48:43.526850 8994 tablet_bootstrap.cc:654] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554: Neither blocks nor log segments found. Creating new log.
I20260811 01:48:43.527082 8996 tablet_bootstrap.cc:492] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4: Bootstrap starting.
I20260811 01:48:43.528512 8994 log.cc:824] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554: Log is configured to *not* fsync() on all Append() calls
I20260811 01:48:43.528952 8997 tablet_bootstrap.cc:492] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2: Bootstrap starting.
I20260811 01:48:43.532236 8996 tablet_bootstrap.cc:654] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4: Neither blocks nor log segments found. Creating new log.
I20260811 01:48:43.532401 8997 tablet_bootstrap.cc:654] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2: Neither blocks nor log segments found. Creating new log.
I20260811 01:48:43.532804 8994 tablet_bootstrap.cc:492] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554: No bootstrap required, opened a new log
I20260811 01:48:43.533098 8994 ts_tablet_manager.cc:1397] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 01:48:43.533584 8997 log.cc:824] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2: Log is configured to *not* fsync() on all Append() calls
I20260811 01:48:43.533638 8996 log.cc:824] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4: Log is configured to *not* fsync() on all Append() calls
I20260811 01:48:43.536520 8997 tablet_bootstrap.cc:492] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2: No bootstrap required, opened a new log
I20260811 01:48:43.536813 8997 ts_tablet_manager.cc:1397] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 01:48:43.539543 8996 tablet_bootstrap.cc:492] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4: No bootstrap required, opened a new log
I20260811 01:48:43.539851 8996 ts_tablet_manager.cc:1397] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.005s
I20260811 01:48:43.543200 8994 raft_consensus.cc:348] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } } peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } }
I20260811 01:48:43.543707 8994 raft_consensus.cc:374] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:48:43.543815 8994 raft_consensus.cc:729] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c86dbfd773a14e1181737ec35c0fc554, State: Initialized, Role: FOLLOWER
I20260811 01:48:43.544518 8994 consensus_queue.cc:260] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } } peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } }
I20260811 01:48:43.545171 8997 raft_consensus.cc:348] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } } peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } }
I20260811 01:48:43.545674 8997 raft_consensus.cc:374] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:48:43.545783 8997 raft_consensus.cc:729] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27c87199ae444777be34b8ce818f1ac2, State: Initialized, Role: FOLLOWER
I20260811 01:48:43.546597 8994 ts_tablet_manager.cc:1428] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260811 01:48:43.546514 8997 consensus_queue.cc:260] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } } peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } }
I20260811 01:48:43.547858 8821 heartbeater.cc:503] Master 127.21.214.126:40013 was elected leader, sending a full tablet report...
I20260811 01:48:43.548523 8997 ts_tablet_manager.cc:1428] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 01:48:43.551457 8996 raft_consensus.cc:348] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } } peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } }
I20260811 01:48:43.551985 8996 raft_consensus.cc:374] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:48:43.552112 8996 raft_consensus.cc:729] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 704161483bfb45e2a571fa19ce99e7f4, State: Initialized, Role: FOLLOWER
I20260811 01:48:43.552806 8996 consensus_queue.cc:260] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } } peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } }
I20260811 01:48:43.554028 8972 heartbeater.cc:503] Master 127.21.214.126:40013 was elected leader, sending a full tablet report...
I20260811 01:48:43.554646 8996 ts_tablet_manager.cc:1428] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
W20260811 01:48:43.600625 8665 tablet.cc:2369] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:48:43.613266 8822 tablet.cc:2369] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:48:43.641990 8974 tablet.cc:2369] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:48:43.753602 9002 raft_consensus.cc:482] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:48:43.754073 9002 raft_consensus.cc:504] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } } peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } }
I20260811 01:48:43.756240 9002 leader_election.cc:288] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c86dbfd773a14e1181737ec35c0fc554 (127.21.214.65:40993), 704161483bfb45e2a571fa19ce99e7f4 (127.21.214.67:40965)
I20260811 01:48:43.764211 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:43.756680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:33677 (local address 127.21.214.65:40993)
0811 01:48:43.756905 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:43.756914 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:43.757460 (+ 546us) server_negotiation.cc:407] Connection header received
0811 01:48:43.757690 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:43.757695 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:43.757782 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:43.757890 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:48:43.758587 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.759697 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.761608 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.762896 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.763059 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:43.763132 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:43.763756 (+ 624us) server_negotiation.cc:299] Negotiation successful
0811 01:48:43.763898 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 01:48:43.764955 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:43.756612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:40993 (local address 127.21.214.66:33677)
0811 01:48:43.757347 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:43.757364 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 01:48:43.757541 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:48:43.757943 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:48:43.757948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:48:43.757983 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:48:43.758402 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:43.758413 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.759844 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.759849 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:43.761432 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:43.761447 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.763300 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.763306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:43.763488 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:43.763523 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 01:48:43.764455 (+ 932us) client_negotiation.cc:240] Negotiation successful
0811 01:48:43.764602 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":378,"threads_started":1}
I20260811 01:48:43.765944 8990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:43.758811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:44155 (local address 127.21.214.67:40965)
0811 01:48:43.759019 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:43.759027 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:43.759050 (+ 23us) server_negotiation.cc:407] Connection header received
0811 01:48:43.759129 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:43.759132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:43.759187 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:43.759290 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:48:43.760028 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.761180 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.762868 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.763686 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.763869 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:43.763945 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:43.764857 (+ 912us) server_negotiation.cc:299] Negotiation successful
0811 01:48:43.765001 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 01:48:43.766140 8609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0e2654a8a446089289affa209c21e" candidate_uuid: "27c87199ae444777be34b8ce818f1ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c86dbfd773a14e1181737ec35c0fc554" is_pre_election: true
I20260811 01:48:43.766741 8609 raft_consensus.cc:2454] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27c87199ae444777be34b8ce818f1ac2 in term 0.
I20260811 01:48:43.767328 9005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:43.758164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:40965 (local address 127.21.214.66:44155)
0811 01:48:43.758858 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:43.758878 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 01:48:43.758998 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:48:43.759311 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:48:43.759315 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:48:43.759339 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:48:43.759830 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:43.759843 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.761325 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.761333 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:43.762703 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:48:43.762717 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.764098 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.764103 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:48:43.764267 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:43.764661 (+ 394us) client_negotiation.cc:769] Sending connection context
0811 01:48:43.766657 (+ 1996us) client_negotiation.cc:240] Negotiation successful
0811 01:48:43.766812 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":1244,"threads_started":1}
I20260811 01:48:43.767549 8688 leader_election.cc:302] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27c87199ae444777be34b8ce818f1ac2, c86dbfd773a14e1181737ec35c0fc554; no voters:
I20260811 01:48:43.767999 9002 raft_consensus.cc:2781] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:48:43.768189 9002 raft_consensus.cc:482] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:48:43.768343 9002 raft_consensus.cc:3037] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:48:43.768496 8923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0e2654a8a446089289affa209c21e" candidate_uuid: "27c87199ae444777be34b8ce818f1ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "704161483bfb45e2a571fa19ce99e7f4" is_pre_election: true
I20260811 01:48:43.769032 8923 raft_consensus.cc:2454] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27c87199ae444777be34b8ce818f1ac2 in term 0.
I20260811 01:48:43.770936 9002 raft_consensus.cc:504] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } } peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } }
I20260811 01:48:43.771595 9002 leader_election.cc:288] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [CANDIDATE]: Term 1 election: Requested vote from peers c86dbfd773a14e1181737ec35c0fc554 (127.21.214.65:40993), 704161483bfb45e2a571fa19ce99e7f4 (127.21.214.67:40965)
I20260811 01:48:43.772102 8609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0e2654a8a446089289affa209c21e" candidate_uuid: "27c87199ae444777be34b8ce818f1ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c86dbfd773a14e1181737ec35c0fc554"
I20260811 01:48:43.772158 8923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e0e2654a8a446089289affa209c21e" candidate_uuid: "27c87199ae444777be34b8ce818f1ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "704161483bfb45e2a571fa19ce99e7f4"
I20260811 01:48:43.772357 8609 raft_consensus.cc:3037] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:48:43.772398 8923 raft_consensus.cc:3037] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:48:43.774735 8609 raft_consensus.cc:2454] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27c87199ae444777be34b8ce818f1ac2 in term 1.
I20260811 01:48:43.775208 8688 leader_election.cc:302] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27c87199ae444777be34b8ce818f1ac2, c86dbfd773a14e1181737ec35c0fc554; no voters:
I20260811 01:48:43.775555 9002 raft_consensus.cc:2781] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:48:43.775740 8923 raft_consensus.cc:2454] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27c87199ae444777be34b8ce818f1ac2 in term 1.
I20260811 01:48:43.777802 9002 raft_consensus.cc:686] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [term 1 LEADER]: Becoming Leader. State: Replica: 27c87199ae444777be34b8ce818f1ac2, State: Running, Role: LEADER
I20260811 01:48:43.778316 9002 consensus_queue.cc:237] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } } peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } }
I20260811 01:48:43.783167 8442 catalog_manager.cc:5495] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27c87199ae444777be34b8ce818f1ac2 (127.21.214.66). New cstate: current_term: 1 leader_uuid: "27c87199ae444777be34b8ce818f1ac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27c87199ae444777be34b8ce818f1ac2" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 40303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 } health_report { overall_health: UNKNOWN } } }
I20260811 01:48:43.808194 8993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:43.802973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58574 (local address 127.21.214.66:40303)
0811 01:48:43.803173 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:43.803178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:43.803197 (+ 19us) server_negotiation.cc:407] Connection header received
0811 01:48:43.803316 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:43.803319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:43.803412 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:43.803549 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:48:43.803878 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.804598 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.806076 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.806511 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.806610 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:43.807098 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:48:43.807652 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:48:43.807768 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:43.807864 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 01:48:43.807937 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 01:48:43.817731 8923 raft_consensus.cc:1261] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4 [term 1 FOLLOWER]: Refusing update from remote peer 27c87199ae444777be34b8ce818f1ac2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:48:43.817785 8609 raft_consensus.cc:1261] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554 [term 1 FOLLOWER]: Refusing update from remote peer 27c87199ae444777be34b8ce818f1ac2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:48:43.818794 9006 consensus_queue.cc:1035] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c86dbfd773a14e1181737ec35c0fc554" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 40993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:48:43.819181 9002 consensus_queue.cc:1035] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "704161483bfb45e2a571fa19ce99e7f4" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 40965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:48:43.830862 9010 mvcc.cc:205] Tried to move back new op lower bound from 7317147335945900032 to 7317147335799910400. Current Snapshot: MvccSnapshot[applied={T|T < 7317147335945900032}]
I20260811 01:48:43.848318 8443 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:48:43.848610 8443 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:48:43.853225 8443 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: Sending DeleteTablet for 3 replicas of tablet 64e0e2654a8a446089289affa209c21e
I20260811 01:48:43.854640 8744 tablet_service.cc:1514] Processing DeleteTablet for tablet 64e0e2654a8a446089289affa209c21e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:48:43 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:58564
I20260811 01:48:43.854773 8901 tablet_service.cc:1514] Processing DeleteTablet for tablet 64e0e2654a8a446089289affa209c21e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:48:43 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:57478
I20260811 01:48:43.854892 8585 tablet_service.cc:1514] Processing DeleteTablet for tablet 64e0e2654a8a446089289affa209c21e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:48:43 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:50926
I20260811 01:48:43.856456 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 8486
I20260811 01:48:43.870771 9026 tablet_replica.cc:330] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554: stopping tablet replica
I20260811 01:48:43.871249 9026 raft_consensus.cc:2229] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:48:43.871908 9026 raft_consensus.cc:2258] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:48:43.873311 9025 tablet_replica.cc:330] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4: stopping tablet replica
I20260811 01:48:43.873875 9025 raft_consensus.cc:2229] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:48:43.874411 9025 raft_consensus.cc:2258] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:48:43.875054 9026 ts_tablet_manager.cc:1905] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:48:43.876721 9025 ts_tablet_manager.cc:1905] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:48:43.879438 9024 tablet_replica.cc:330] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2: stopping tablet replica
I20260811 01:48:43.881538 9024 raft_consensus.cc:2229] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:48:43.882710 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:43.857511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50936 (local address 127.21.214.65:40993)
0811 01:48:43.858188 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:43.858195 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:43.858214 (+ 19us) server_negotiation.cc:407] Connection header received
0811 01:48:43.858274 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:43.858280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:43.858371 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:43.858474 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:43.861025 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.862365 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.864660 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:43.865457 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:43.865577 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:43.871728 (+ 6151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:43.871742 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:43.871760 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:43.871810 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:43.877520 (+ 5710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:43.878658 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:43.878668 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:43.878671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:43.878858 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:43.879636 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:43.879642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:43.879646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:43.880159 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:43.880519 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:43.881935 (+ 1416us) server_negotiation.cc:299] Negotiation successful
0811 01:48:43.882243 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545}
I20260811 01:48:43.883746 9026 ts_tablet_manager.cc:1918] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:48:43.883927 9026 log.cc:1196] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/64e0e2654a8a446089289affa209c21e
I20260811 01:48:43.883898 9025 ts_tablet_manager.cc:1918] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:48:43.884068 9025 log.cc:1196] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/64e0e2654a8a446089289affa209c21e
I20260811 01:48:43.884557 9026 ts_tablet_manager.cc:1939] T 64e0e2654a8a446089289affa209c21e P c86dbfd773a14e1181737ec35c0fc554: Deleting consensus metadata
I20260811 01:48:43.884630 9025 ts_tablet_manager.cc:1939] T 64e0e2654a8a446089289affa209c21e P 704161483bfb45e2a571fa19ce99e7f4: Deleting consensus metadata
I20260811 01:48:43.884712 8657 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:48:43.885370 9024 raft_consensus.cc:2258] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:48:43.886924 8426 catalog_manager.cc:4841] TS 704161483bfb45e2a571fa19ce99e7f4 (127.21.214.67:40965): tablet 64e0e2654a8a446089289affa209c21e (table test-table [id=e2c89c87de7a4e6aa4547c569a8cc725]) successfully deleted
I20260811 01:48:43.887544 9024 ts_tablet_manager.cc:1905] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:48:43.899667 9024 ts_tablet_manager.cc:1918] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:48:43.899885 9024 log.cc:1196] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/64e0e2654a8a446089289affa209c21e
I20260811 01:48:43.900561 9024 ts_tablet_manager.cc:1939] T 64e0e2654a8a446089289affa209c21e P 27c87199ae444777be34b8ce818f1ac2: Deleting consensus metadata
I20260811 01:48:43.905282 8425 catalog_manager.cc:4841] TS 27c87199ae444777be34b8ce818f1ac2 (127.21.214.66:40303): tablet 64e0e2654a8a446089289affa209c21e (table test-table [id=e2c89c87de7a4e6aa4547c569a8cc725]) successfully deleted
I20260811 01:48:44.027189 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 8486
I20260811 01:48:44.055658 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 8669
W20260811 01:48:44.055641 8425 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.214.65:40993 (error 108)
I20260811 01:48:44.058188 9029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:44.057349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:40993 (local address 127.0.0.1:50940)
0811 01:48:44.057815 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:44.057955 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":220,"threads_started":1}
W20260811 01:48:44.058590 8425 catalog_manager.cc:4584] TS c86dbfd773a14e1181737ec35c0fc554 (127.21.214.65:40993): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 64e0e2654a8a446089289affa209c21e: Network error: Client connection negotiation failed: client connection to 127.21.214.65:40993: connect: Connection refused (error 111)
I20260811 01:48:44.073846 8993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:44.056483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58578 (local address 127.21.214.66:40303)
0811 01:48:44.056681 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:44.056687 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:44.057273 (+ 586us) server_negotiation.cc:407] Connection header received
0811 01:48:44.057377 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:44.057382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:44.057511 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:44.057636 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:44.059503 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:44.060788 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:44.062323 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:44.063072 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:44.063214 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:44.066193 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:44.066214 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:44.066237 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:44.066294 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:44.070446 (+ 4152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:44.071279 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:44.071291 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:44.071295 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:44.071476 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:44.072002 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:44.072008 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:44.072011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:44.072493 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:44.072860 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:44.073129 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 01:48:44.073407 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 01:48:44.074731 8816 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:48:44.112481 9029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:44.111996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:40993 (local address 127.0.0.1:50946)
0811 01:48:44.112223 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:44.112336 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 01:48:44.113725 9029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:44.113292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:40993 (local address 127.0.0.1:50962)
0811 01:48:44.113473 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:44.113542 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 01:48:44.174217 9029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:44.173719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:40993 (local address 127.0.0.1:50970)
0811 01:48:44.173961 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:44.174064 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 01:48:44.175408 9029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:44.175003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:40993 (local address 127.0.0.1:50984)
0811 01:48:44.175177 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:44.175251 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 01:48:44.194119 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 8669
I20260811 01:48:44.218745 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 8824
I20260811 01:48:44.234144 8990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:44.219679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57484 (local address 127.21.214.67:40965)
0811 01:48:44.219928 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 01:48:44.219935 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:48:44.220022 (+ 87us) server_negotiation.cc:407] Connection header received
0811 01:48:44.220271 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:48:44.220275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:48:44.220348 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:48:44.220460 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:48:44.222090 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:44.222974 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:44.224478 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:48:44.224890 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:48:44.225032 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:48:44.227972 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:48:44.227989 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:48:44.228008 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:48:44.228056 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:48:44.231143 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:44.231947 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:44.231955 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:44.231957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:44.232112 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:48:44.232479 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:48:44.232483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:48:44.232485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:48:44.232873 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:48:44.233105 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 01:48:44.233460 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 01:48:44.233643 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 01:48:44.234884 8966 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:48:44.255940 9029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:44.255455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:40993 (local address 127.0.0.1:50994)
0811 01:48:44.255674 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:44.255768 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 01:48:44.257120 9029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:44.256686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:40993 (local address 127.0.0.1:50996)
0811 01:48:44.256889 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:44.256973 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 01:48:44.330003 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 8824
I20260811 01:48:44.353860 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 8406
I20260811 01:48:44.355176 8478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:48:44.475390 9029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:48:44.474825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:40993 (local address 127.0.0.1:51004)
0811 01:48:44.475044 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 01:48:44.475188 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 01:48:44.475746 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 8406
I20260811 01:49:04.506312 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:40013
--webserver_interface=127.21.214.126
--webserver_port=35031
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:40013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:49:04.882773 9033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:04.883100 9033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:04.883248 9033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:04.890878 9033 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 01:49:04.890990 9033 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 01:49:04.891029 9033 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 01:49:04.891144 9033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:49:04.891251 9033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:04.891384 9033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:04.891477 9033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:49:04.891563 9033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:49:04.902136 9033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:40013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:40013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=35031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:04.903697 9033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:04.905398 9033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:04.918256 9038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:04.918290 9041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:04.918357 9039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:04.919909 9033 server_base.cc:1034] running on GCE node
I20260811 01:49:04.920933 9033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:04.922619 9033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:04.924032 9033 hybrid_clock.cc:648] HybridClock initialized: now 1786412944923925 us; error 75 us; skew 500 ppm
I20260811 01:49:04.924597 9033 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:04 dist-test-slave-9gcw krb5kdc[7756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412944, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:04.929543 9033 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:49:04.931545 9033 webserver.cc:460] Webserver started at http://127.21.214.126:35031/ using document root <none> and password file <none>
I20260811 01:49:04.932328 9033 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:04.932425 9033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:04.938445 9033 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 01:49:04.941643 9048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:04.943197 9033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 01:49:04.943404 9033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "fc0a298e73dd4848ae307d38c888ded9"
format_stamp: "Formatted at 2026-08-11 01:48:17 on dist-test-slave-9gcw"
server_key: "10087ef2ea7922aab8b3efddba6985ef"
server_key_iv: "317d1cde71154733a98094111cb40be4"
server_key_version: "encryptionkey@0"
I20260811 01:49:04.944494 9033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:04.975306 9033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:04.980261 9033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:04.980585 9033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:05.003542 9033 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:40013
I20260811 01:49:05.003580 9107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:40013 every 8 connection(s)
I20260811 01:49:05.005441 9033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:49:05.012660 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 9033
I20260811 01:49:05.014007 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:40993
--local_ip_for_outbound_sockets=127.21.214.65
--tserver_master_addrs=127.21.214.126:40013
--webserver_port=46845
--webserver_interface=127.21.214.65
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:49:05.014822 9108 sys_catalog.cc:263] Verifying existing consensus state
I20260811 01:49:05.020869 9108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: Bootstrap starting.
I20260811 01:49:05.046555 9108 log.cc:824] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:05.066535 9108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 01:49:05.067251 9108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: Bootstrap complete.
I20260811 01:49:05.078312 9108 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0a298e73dd4848ae307d38c888ded9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40013 } }
I20260811 01:49:05.079514 9108 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0a298e73dd4848ae307d38c888ded9, State: Initialized, Role: FOLLOWER
I20260811 01:49:05.080377 9108 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0a298e73dd4848ae307d38c888ded9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40013 } }
I20260811 01:49:05.080668 9108 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:49:05.080827 9108 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:49:05.081038 9108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 2 FOLLOWER]: Advancing to term 3
I20260811 01:49:05.085611 9108 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0a298e73dd4848ae307d38c888ded9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40013 } }
I20260811 01:49:05.086180 9108 leader_election.cc:302] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc0a298e73dd4848ae307d38c888ded9; no voters:
I20260811 01:49:05.086946 9108 leader_election.cc:288] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 01:49:05.087123 9114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 01:49:05.088338 9114 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [term 3 LEADER]: Becoming Leader. State: Replica: fc0a298e73dd4848ae307d38c888ded9, State: Running, Role: LEADER
I20260811 01:49:05.088982 9114 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0a298e73dd4848ae307d38c888ded9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40013 } }
I20260811 01:49:05.089560 9108 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:49:05.092176 9115 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "fc0a298e73dd4848ae307d38c888ded9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0a298e73dd4848ae307d38c888ded9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40013 } } }
I20260811 01:49:05.093043 9115 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:05.092257 9116 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc0a298e73dd4848ae307d38c888ded9. Latest consensus state: current_term: 3 leader_uuid: "fc0a298e73dd4848ae307d38c888ded9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc0a298e73dd4848ae307d38c888ded9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40013 } } }
I20260811 01:49:05.093442 9116 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9 [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:05.095641 9123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:49:05.110179 9123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:49:05.112084 9123 catalog_manager.cc:1259] Loaded cluster ID: d7f0f9cd8c9a43daa9374df302479ef7
I20260811 01:49:05.112207 9123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:49:05.118014 9123 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:49:05.120271 9123 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: Loaded TSK: 0
I20260811 01:49:05.120456 9123 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fc0a298e73dd4848ae307d38c888ded9: Loaded TSK: 1
F20260811 01:49:05.121894 9123 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f416321d9ad google::LogMessage::Fail() at ??:0
@ 0x7f416322088a google::LogMessage::SendToLog() at ??:0
@ 0x7f416321d4de google::LogMessage::Flush() at ??:0
@ 0x7f416321ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4166f817d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f4166f7073e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f4178c4c1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f4178c4b783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f4178cd72a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f4178cd70c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f4178d215e9 std::function<>::operator()() at ??:0
@ 0x7f4178c4fcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f4178c4dc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4178cd5610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f4178cd5439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4165e69345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4165e6c91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f4165e6c799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4165e47eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f41691216db start_thread at ??:0
@ 0x7f41620b761f clone at ??:0
W20260811 01:49:05.468374 9110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:05.468714 9110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:05.468896 9110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:05.476823 9110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:05.476956 9110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:05.477180 9110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:49:05.487607 9110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:40993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=46845
--tserver_master_addrs=127.21.214.126:40013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:05.489073 9110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:05.490563 9110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:05.505277 9163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:05.505441 9166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:05.505918 9164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:05.507210 9110 server_base.cc:1034] running on GCE node
I20260811 01:49:05.507977 9110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:05.509428 9110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:05.510717 9110 hybrid_clock.cc:648] HybridClock initialized: now 1786412945510659 us; error 58 us; skew 500 ppm
I20260811 01:49:05.511111 9110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:05 dist-test-slave-9gcw krb5kdc[7756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412945, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:05.514923 9110 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:49:05.516697 9110 webserver.cc:460] Webserver started at http://127.21.214.65:46845/ using document root <none> and password file <none>
I20260811 01:49:05.517549 9110 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:05.517658 9110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:05.523794 9110 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 01:49:05.527230 9176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:05.528982 9110 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 01:49:05.529265 9110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "c86dbfd773a14e1181737ec35c0fc554"
format_stamp: "Formatted at 2026-08-11 01:48:18 on dist-test-slave-9gcw"
server_key: "25260aa86b4b07b8d9c35cd5bf02c4c9"
server_key_iv: "954e7a3c2d74fb2d465c9a0079c9aa28"
server_key_version: "encryptionkey@0"
I20260811 01:49:05.530422 9110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:05.547235 9110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:05.552117 9110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:05.552469 9110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:05.553722 9110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:05.555234 9110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:05.555382 9110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:05.555526 9110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:05.555660 9110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:05.594619 9110 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:40993
I20260811 01:49:05.594786 9306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:40993 every 8 connection(s)
I20260811 01:49:05.596339 9110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:49:05.601068 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 9110
I20260811 01:49:05.601940 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:40303
--local_ip_for_outbound_sockets=127.21.214.66
--tserver_master_addrs=127.21.214.126:40013
--webserver_port=36197
--webserver_interface=127.21.214.66
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:49:05.601990 9310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:05.599753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:53849)
0811 01:49:05.600554 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:05.600826 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":305,"threads_started":1}
W20260811 01:49:05.602950 9307 heartbeater.cc:644] Failed to heartbeat to 127.21.214.126:40013 (0 consecutive failures): Network error: Failed to ping master at 127.21.214.126:40013: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
I20260811 01:49:05.604007 9310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:05.603585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:44187)
0811 01:49:05.603771 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:05.603870 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 01:49:05.605038 9310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:05.604645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:55689)
0811 01:49:05.604824 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:05.604906 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 01:49:05.605383 9307 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 01:49:05.965700 9311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:05.966025 9311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:05.966126 9311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:05.973513 9311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:05.973649 9311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:05.973770 9311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:49:05.983788 9311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:40303
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=36197
--tserver_master_addrs=127.21.214.126:40013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:05.985131 9311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:05.986539 9311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:06.001000 9319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:06.001116 9316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:06.001116 9317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:06.002784 9311 server_base.cc:1034] running on GCE node
I20260811 01:49:06.003448 9311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:06.004423 9311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:06.005690 9311 hybrid_clock.cc:648] HybridClock initialized: now 1786412946005632 us; error 67 us; skew 500 ppm
I20260811 01:49:06.006050 9311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:06 dist-test-slave-9gcw krb5kdc[7756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412946, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:06.009800 9311 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:49:06.011492 9311 webserver.cc:460] Webserver started at http://127.21.214.66:36197/ using document root <none> and password file <none>
I20260811 01:49:06.012256 9311 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:06.012427 9311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:06.018652 9311 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 01:49:06.021710 9326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:06.023170 9311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 01:49:06.023377 9311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "27c87199ae444777be34b8ce818f1ac2"
format_stamp: "Formatted at 2026-08-11 01:48:19 on dist-test-slave-9gcw"
server_key: "44df448c0f3782a0c495cf278822c28a"
server_key_iv: "252f47ffff186787770c7efc623d64af"
server_key_version: "encryptionkey@0"
I20260811 01:49:06.024442 9311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:06.044989 9311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:06.050026 9311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:06.050354 9311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:06.051462 9311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:06.052949 9311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:06.053109 9311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:06.053324 9311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:06.053421 9311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:06.090119 9311 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:40303
I20260811 01:49:06.090246 9460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:40303 every 8 connection(s)
I20260811 01:49:06.091636 9311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:49:06.096199 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 9311
I20260811 01:49:06.097071 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:40965
--local_ip_for_outbound_sockets=127.21.214.67
--tserver_master_addrs=127.21.214.126:40013
--webserver_port=39387
--webserver_interface=127.21.214.67
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:49:06.097314 9463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:06.094895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:40537)
0811 01:49:06.095664 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:06.095990 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":289,"threads_started":1}
W20260811 01:49:06.098086 9461 heartbeater.cc:644] Failed to heartbeat to 127.21.214.126:40013 (0 consecutive failures): Network error: Failed to ping master at 127.21.214.126:40013: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
I20260811 01:49:06.099067 9463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:06.098736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:58657)
0811 01:49:06.098889 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:06.098958 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 01:49:06.100030 9463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:06.099707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:43297)
0811 01:49:06.099845 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:06.099920 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 01:49:06.100302 9461 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 01:49:06.454056 9464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:06.454375 9464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:06.454545 9464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:06.462136 9464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:06.462268 9464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:06.462466 9464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:49:06.472970 9464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:40965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=39387
--tserver_master_addrs=127.21.214.126:40013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:06.474454 9464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:06.475873 9464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:06.490175 9469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:06.490260 9470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:06.490260 9472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:06.491495 9464 server_base.cc:1034] running on GCE node
I20260811 01:49:06.492174 9464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:06.493270 9464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:06.494494 9464 hybrid_clock.cc:648] HybridClock initialized: now 1786412946494433 us; error 47 us; skew 500 ppm
I20260811 01:49:06.494850 9464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:06 dist-test-slave-9gcw krb5kdc[7756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412946, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:06.498402 9464 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:49:06.500079 9464 webserver.cc:460] Webserver started at http://127.21.214.67:39387/ using document root <none> and password file <none>
I20260811 01:49:06.500917 9464 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:06.501029 9464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:06.506784 9464 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 01:49:06.509864 9480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:06.511380 9464 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:49:06.511530 9464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "704161483bfb45e2a571fa19ce99e7f4"
format_stamp: "Formatted at 2026-08-11 01:48:19 on dist-test-slave-9gcw"
server_key: "82a8861f769c6a7f85d1b337619c2497"
server_key_iv: "a1097fee717e69b6e422853bf81a3181"
server_key_version: "encryptionkey@0"
I20260811 01:49:06.512569 9464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:06.536528 9464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:06.541483 9464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:06.541741 9464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:06.542754 9464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:06.544224 9464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:06.544324 9464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:06.544409 9464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:06.544456 9464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:06.581491 9464 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:40965
I20260811 01:49:06.581548 9609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:40965 every 8 connection(s)
I20260811 01:49:06.583042 9464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:49:06.588270 9614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:06.586311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:47629)
0811 01:49:06.587109 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:06.587380 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":304,"threads_started":1}
W20260811 01:49:06.589031 9610 heartbeater.cc:644] Failed to heartbeat to 127.21.214.126:40013 (0 consecutive failures): Network error: Failed to ping master at 127.21.214.126:40013: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
I20260811 01:49:06.589980 9614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:06.589625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:59921)
0811 01:49:06.589794 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:06.589873 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 01:49:06.590345 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 9464
I20260811 01:49:06.590939 9614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:06.590597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:58677)
0811 01:49:06.590738 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:06.590830 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 01:49:06.591269 9610 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 01:49:06.606956 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:06.606006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:54421)
0811 01:49:06.606548 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:06.606678 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":276,"threads_started":1}
I20260811 01:49:07.101958 9617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:07.101006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:59183)
0811 01:49:07.101559 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:07.101689 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":292,"threads_started":1}
I20260811 01:49:07.593014 9618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:07.591937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:56945)
0811 01:49:07.592528 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:07.592628 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":311,"threads_started":1}
I20260811 01:49:07.608736 9619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:07.607978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:54327)
0811 01:49:07.608471 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:07.608559 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":252,"threads_started":1}
I20260811 01:49:08.103919 9620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:08.102982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:47861)
0811 01:49:08.103476 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:08.103566 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":241,"threads_started":1}
I20260811 01:49:08.595125 9621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:08.594074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:58389)
0811 01:49:08.594844 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:08.594947 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":423,"threads_started":1}
I20260811 01:49:08.610569 9622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:08.609813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:45793)
0811 01:49:08.610297 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:08.610385 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":257,"threads_started":1}
I20260811 01:49:09.105865 9623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:09.104916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:47571)
0811 01:49:09.105514 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:09.105627 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":304,"threads_started":1}
I20260811 01:49:09.597023 9624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:09.596139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:46191)
0811 01:49:09.596673 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:09.596792 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":262,"threads_started":1}
I20260811 01:49:09.612835 9626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:09.611851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:40245)
0811 01:49:09.612511 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:09.612621 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":368,"threads_started":1}
I20260811 01:49:10.108398 9627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:10.107412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:39083)
0811 01:49:10.107985 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:10.108079 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":324,"threads_started":1}
I20260811 01:49:10.598815 9628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:10.598028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:58559)
0811 01:49:10.598566 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:10.598645 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":325,"threads_started":1}
I20260811 01:49:10.615062 9629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:10.613944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:38919)
0811 01:49:10.614713 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:10.614816 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":383,"threads_started":1}
I20260811 01:49:11.110519 9631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:11.109648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:33597)
0811 01:49:11.110221 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:11.110333 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":317,"threads_started":1}
I20260811 01:49:11.600786 9632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:11.599954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:55625)
0811 01:49:11.600534 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:11.600616 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":360,"threads_started":1}
I20260811 01:49:11.617314 9634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:11.616269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:52613)
0811 01:49:11.616913 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:11.617016 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":304,"threads_started":1}
I20260811 01:49:12.112344 9635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:12.111592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:47055)
0811 01:49:12.112064 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:12.112165 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":240,"threads_started":1}
I20260811 01:49:12.602912 9636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:12.601974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:56927)
0811 01:49:12.602538 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:12.602639 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":325,"threads_started":1}
I20260811 01:49:12.619150 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:12.618316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:59991)
0811 01:49:12.618847 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:12.618932 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":267,"threads_started":1}
I20260811 01:49:13.114462 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:13.113481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:39597)
0811 01:49:13.114005 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:13.114200 (+ 195us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":250,"threads_started":1}
I20260811 01:49:13.604866 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:13.603972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:43843)
0811 01:49:13.604558 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:13.604639 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":315,"threads_started":1}
I20260811 01:49:13.620962 9640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:13.620183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:35359)
0811 01:49:13.620636 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:13.620786 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":232,"threads_started":1}
I20260811 01:49:14.116760 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:14.115694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:56063)
0811 01:49:14.116278 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:14.116399 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":263,"threads_started":1}
I20260811 01:49:14.606595 9642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:14.605805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:36699)
0811 01:49:14.606344 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:14.606429 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":268,"threads_started":1}
I20260811 01:49:14.622747 9643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:14.621923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:41921)
0811 01:49:14.622473 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:14.622561 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":280,"threads_started":1}
I20260811 01:49:15.118952 9644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:15.117992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:48771)
0811 01:49:15.118610 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:15.118763 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":344,"threads_started":1}
I20260811 01:49:15.608536 9645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:15.607669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:43853)
0811 01:49:15.608252 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:15.608348 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":298,"threads_started":1}
I20260811 01:49:15.624711 9646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:15.623799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:60383)
0811 01:49:15.624356 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:15.624463 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":286,"threads_started":1}
I20260811 01:49:16.120826 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:16.120040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:53459)
0811 01:49:16.120556 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:16.120650 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":292,"threads_started":1}
I20260811 01:49:16.610673 9648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:16.609859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:51827)
0811 01:49:16.610413 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:16.610508 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":295,"threads_started":1}
I20260811 01:49:16.626711 9649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:16.625882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:39503)
0811 01:49:16.626405 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:16.626510 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":276,"threads_started":1}
I20260811 01:49:17.122792 9651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:17.121934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:40903)
0811 01:49:17.122469 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:17.122578 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":285,"threads_started":1}
I20260811 01:49:17.612990 9652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:17.612030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:51879)
0811 01:49:17.612645 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:17.612773 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":240,"threads_started":1}
I20260811 01:49:17.628933 9653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:17.627983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:55183)
0811 01:49:17.628523 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:17.628694 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":290,"threads_started":1}
I20260811 01:49:18.125059 9654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:18.124094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:58313)
0811 01:49:18.124701 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:18.124801 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":330,"threads_started":1}
I20260811 01:49:18.615052 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:18.614140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:50931)
0811 01:49:18.614774 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:18.614876 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":292,"threads_started":1}
I20260811 01:49:18.630956 9656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:18.629986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:38113)
0811 01:49:18.630551 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:18.630717 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":295,"threads_started":1}
I20260811 01:49:19.127403 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:19.126236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:51019)
0811 01:49:19.126882 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:19.127030 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":303,"threads_started":1}
I20260811 01:49:19.617309 9659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:19.616282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:41699)
0811 01:49:19.616903 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:19.617014 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":345,"threads_started":1}
I20260811 01:49:19.632858 9660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:19.632044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:39395)
0811 01:49:19.632543 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:19.632646 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":237,"threads_started":1}
I20260811 01:49:20.129606 9661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:20.128552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:46599)
0811 01:49:20.129098 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:20.129348 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":294,"threads_started":1}
I20260811 01:49:20.619259 9662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:20.618400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:45921)
0811 01:49:20.618960 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:20.619054 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":317,"threads_started":1}
I20260811 01:49:20.635135 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:20.634261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:46517)
0811 01:49:20.634828 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:20.634936 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":305,"threads_started":1}
I20260811 01:49:21.131644 9664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:21.130880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.66:51175)
0811 01:49:21.131349 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:21.131452 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":229,"threads_started":1}
I20260811 01:49:21.596966 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 9110
I20260811 01:49:21.621600 9666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:21.620626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.67:33325)
0811 01:49:21.621210 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:21.621340 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":295,"threads_started":1}
I20260811 01:49:21.622853 9665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:21.598368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41950 (local address 127.21.214.65:40993)
0811 01:49:21.599714 (+ 1346us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:21.599731 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:21.599774 (+ 43us) server_negotiation.cc:407] Connection header received
0811 01:49:21.601710 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:21.601721 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:21.602195 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:21.602577 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:21.604267 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:21.605637 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:21.606935 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:21.607619 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:21.607832 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:21.615578 (+ 7746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:21.615587 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:21.615606 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:21.615647 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:21.619057 (+ 3410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:21.619885 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:21.619893 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:21.619902 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:21.620091 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:21.620612 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:21.620616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:21.620618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:21.621320 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:21.621576 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:21.622180 (+ 604us) server_negotiation.cc:299] Negotiation successful
0811 01:49:21.622568 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1171,"thread_start_us":1035,"threads_started":1}
I20260811 01:49:21.624464 9301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:21.715134 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 9110
I20260811 01:49:21.715188 9668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:21.713859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40013 (local address 127.21.214.65:55809)
0811 01:49:21.714436 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:21.714811 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.126:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":253,"threads_started":1}
I20260811 01:49:21.740969 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 9311
I20260811 01:49:21.760938 9669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:21.742234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34584 (local address 127.21.214.66:40303)
0811 01:49:21.742744 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:21.742759 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:21.742801 (+ 42us) server_negotiation.cc:407] Connection header received
0811 01:49:21.744466 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:21.744476 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:21.744938 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:21.745371 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:21.746819 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:21.748172 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:21.749966 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:21.750674 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:21.750823 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:21.753752 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:21.753760 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:21.753778 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:21.753826 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:21.757299 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:21.758234 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:21.758242 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:21.758250 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:21.758460 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:21.758825 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:21.758828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:21.758830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:21.759489 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:21.759758 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:21.760251 (+ 493us) server_negotiation.cc:299] Negotiation successful
0811 01:49:21.760643 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":215,"threads_started":1}
I20260811 01:49:21.762547 9455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:21.851066 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 9311
I20260811 01:49:21.874684 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 9464
I20260811 01:49:21.894277 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:21.875909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33376 (local address 127.21.214.67:40965)
0811 01:49:21.876559 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:21.876575 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:21.876619 (+ 44us) server_negotiation.cc:407] Connection header received
0811 01:49:21.878398 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:21.878408 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:21.878870 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:21.879189 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:21.880584 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:21.882022 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:21.883314 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:21.884003 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:21.884136 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:21.887308 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:21.887316 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:21.887334 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:21.887387 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:21.890623 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:21.891370 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:21.891377 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:21.891385 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:21.891562 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:21.891992 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:21.891996 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:21.891998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:21.892568 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:21.892805 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:21.893417 (+ 612us) server_negotiation.cc:299] Negotiation successful
0811 01:49:21.893925 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":309,"threads_started":1}
I20260811 01:49:21.896063 9605 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:21.986783 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 9464
2026-08-11T01:49:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (66169 ms)
[----------] 30 tests from SecurityITest (325766 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:49:22 dist-test-slave-9gcw krb5kdc[9676](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:49:22 dist-test-slave-9gcw krb5kdc[9676](info): set up 2 sockets
Aug 11 01:49:22 dist-test-slave-9gcw krb5kdc[9676](info): commencing operation
krb5kdc: starting...
W20260811 01:49:23.266535 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:49:23 dist-test-slave-9gcw krb5kdc[9676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:49:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:49:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:49:23.617935 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:40201
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:37865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:40201
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:49:23.997993 9691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:23.998322 9691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:23.998505 9691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:24.006217 9691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:49:24.006351 9691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:24.006455 9691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:24.006552 9691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:49:24.006644 9691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:49:24.017005 9691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:37865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:40201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:40201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:24.018510 9691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:24.019846 9691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:24.032927 9698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:24.032845 9699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:24.032927 9702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:24.033267 9691 server_base.cc:1034] running on GCE node
I20260811 01:49:24.035385 9691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:24.037123 9691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:24.038461 9691 hybrid_clock.cc:648] HybridClock initialized: now 1786412964038402 us; error 53 us; skew 500 ppm
I20260811 01:49:24.038944 9691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:24 dist-test-slave-9gcw krb5kdc[9676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412964, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:24.042649 9691 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:49:24.044431 9691 webserver.cc:460] Webserver started at http://127.21.214.126:40465/ using document root <none> and password file <none>
I20260811 01:49:24.045336 9691 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:24.045504 9691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:24.045864 9691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:24.049353 9691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "23d3eb51c7c342199836510d06e09a73"
format_stamp: "Formatted at 2026-08-11 01:49:24 on dist-test-slave-9gcw"
server_key: "7053155bdfcb25fe288ac97f2e53a6f1"
server_key_iv: "1b7f3b9c0890abf47837bcff01bda8ce"
server_key_version: "encryptionkey@0"
I20260811 01:49:24.050236 9691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "23d3eb51c7c342199836510d06e09a73"
format_stamp: "Formatted at 2026-08-11 01:49:24 on dist-test-slave-9gcw"
server_key: "7053155bdfcb25fe288ac97f2e53a6f1"
server_key_iv: "1b7f3b9c0890abf47837bcff01bda8ce"
server_key_version: "encryptionkey@0"
I20260811 01:49:24.056483 9691 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 01:49:24.061112 9711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:24.062820 9691 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 01:49:24.063046 9691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "23d3eb51c7c342199836510d06e09a73"
format_stamp: "Formatted at 2026-08-11 01:49:24 on dist-test-slave-9gcw"
server_key: "7053155bdfcb25fe288ac97f2e53a6f1"
server_key_iv: "1b7f3b9c0890abf47837bcff01bda8ce"
server_key_version: "encryptionkey@0"
I20260811 01:49:24.063313 9691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:24.081527 9691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:24.086563 9691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:24.086968 9691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:24.109030 9691 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:40201
I20260811 01:49:24.109030 9768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:40201 every 8 connection(s)
I20260811 01:49:24.110766 9691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:49:24.114106 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 9691
I20260811 01:49:24.114435 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:49:24.115291 22361 external_mini_cluster.cc:1428] Setting key 5a793f71f5e10fd402a0e35504798cdb
I20260811 01:49:24.115290 9769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:24.132006 9769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73: Bootstrap starting.
Aug 11 01:49:24 dist-test-slave-9gcw krb5kdc[9676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:49:24.136917 9769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:24.138761 9769 log.cc:824] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:24.144464 9769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73: No bootstrap required, opened a new log
I20260811 01:49:24.147064 9772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:24.117713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50666 (local address 127.21.214.126:40201)
0811 01:49:24.118677 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:24.118695 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:24.118754 (+ 59us) server_negotiation.cc:407] Connection header received
0811 01:49:24.120772 (+ 2018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:24.120815 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:24.121431 (+ 616us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:24.121938 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:24.123522 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:24.125561 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:24.127706 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:24.128919 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:24.129139 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:24.135004 (+ 5865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:24.135020 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:24.135045 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:24.135152 (+ 107us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:24.140634 (+ 5482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:24.141950 (+ 1316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:24.141961 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:24.141974 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:24.142236 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:24.142857 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:24.142863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:24.142866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:24.143756 (+ 890us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:24.144104 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:24.144803 (+ 699us) server_negotiation.cc:299] Negotiation successful
0811 01:49:24.145436 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":404,"threads_started":1}
I20260811 01:49:24.154997 9769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23d3eb51c7c342199836510d06e09a73" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40201 } }
I20260811 01:49:24.155534 9769 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:24.155670 9769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23d3eb51c7c342199836510d06e09a73, State: Initialized, Role: FOLLOWER
I20260811 01:49:24.156486 9769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23d3eb51c7c342199836510d06e09a73" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40201 } }
I20260811 01:49:24.156778 9769 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:49:24.156948 9769 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:49:24.157233 9769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:24.159864 9769 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23d3eb51c7c342199836510d06e09a73" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40201 } }
I20260811 01:49:24.160563 9769 leader_election.cc:302] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23d3eb51c7c342199836510d06e09a73; no voters:
I20260811 01:49:24.161432 9769 leader_election.cc:288] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:49:24.161664 9774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:49:24.162500 9774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73 [term 1 LEADER]: Becoming Leader. State: Replica: 23d3eb51c7c342199836510d06e09a73, State: Running, Role: LEADER
I20260811 01:49:24.163106 9774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23d3eb51c7c342199836510d06e09a73" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40201 } }
I20260811 01:49:24.164044 9769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:49:24.166465 9777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23d3eb51c7c342199836510d06e09a73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23d3eb51c7c342199836510d06e09a73" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40201 } } }
I20260811 01:49:24.166857 9777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73 [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:24.166527 9776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23d3eb51c7c342199836510d06e09a73. Latest consensus state: current_term: 1 leader_uuid: "23d3eb51c7c342199836510d06e09a73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23d3eb51c7c342199836510d06e09a73" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40201 } } }
I20260811 01:49:24.166980 9776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73 [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:24.171386 9781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:49:24.177018 9781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:49:24.186357 9781 catalog_manager.cc:1347] Generated new cluster ID: 353ef4c34bd543bda077d0f29ac8b357
I20260811 01:49:24.186676 9781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:49:24.198050 9781 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:49:24.199554 9781 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:49:24.216961 9781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73: Generated new TSK 0
I20260811 01:49:24.217995 9781 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
I20260811 01:49:24.297423 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40201
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:37865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:49:24.668798 9798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:24.669119 9798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:24.669299 9798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:24.677011 9798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:24.677197 9798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:24.677351 9798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:49:24.687927 9798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:37865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40201
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:24.689416 9798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:24.690807 9798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:24.705569 9804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:24.705811 9807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:24.705804 9805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:24.706847 9798 server_base.cc:1034] running on GCE node
I20260811 01:49:24.707573 9798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:24.708725 9798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:24.709977 9798 hybrid_clock.cc:648] HybridClock initialized: now 1786412964709910 us; error 56 us; skew 500 ppm
I20260811 01:49:24.710306 9798 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:24 dist-test-slave-9gcw krb5kdc[9676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412964, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:24.713905 9798 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:49:24.715602 9798 webserver.cc:460] Webserver started at http://127.21.214.65:34647/ using document root <none> and password file <none>
I20260811 01:49:24.716423 9798 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:24.716602 9798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:24.716957 9798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:24.720366 9798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "e33ad49a142e434cbeb5ccdfdb4aeada"
format_stamp: "Formatted at 2026-08-11 01:49:24 on dist-test-slave-9gcw"
server_key: "d6d4932cf1522b1d689b69a477be7342"
server_key_iv: "7e70381ead6a3387a301b4bb5175e582"
server_key_version: "encryptionkey@0"
I20260811 01:49:24.721275 9798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "e33ad49a142e434cbeb5ccdfdb4aeada"
format_stamp: "Formatted at 2026-08-11 01:49:24 on dist-test-slave-9gcw"
server_key: "d6d4932cf1522b1d689b69a477be7342"
server_key_iv: "7e70381ead6a3387a301b4bb5175e582"
server_key_version: "encryptionkey@0"
I20260811 01:49:24.727420 9798 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260811 01:49:24.731484 9816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:24.733129 9798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 01:49:24.733409 9798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "e33ad49a142e434cbeb5ccdfdb4aeada"
format_stamp: "Formatted at 2026-08-11 01:49:24 on dist-test-slave-9gcw"
server_key: "d6d4932cf1522b1d689b69a477be7342"
server_key_iv: "7e70381ead6a3387a301b4bb5175e582"
server_key_version: "encryptionkey@0"
I20260811 01:49:24.733641 9798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:24.754401 9798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:24.759210 9798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:24.759621 9798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:24.761030 9798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:24.762642 9798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:24.762756 9798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:24.762953 9798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:24.763031 9798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:24.801764 9798 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:38773
I20260811 01:49:24.801939 9943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:38773 every 8 connection(s)
I20260811 01:49:24.803267 9798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:49:24.811651 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 9798
I20260811 01:49:24.811939 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:49:24.812819 22361 external_mini_cluster.cc:1428] Setting key fcfeb906db78013742b1438e5d945968
Aug 11 01:49:24 dist-test-slave-9gcw krb5kdc[9676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412964, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:49:24.827912 9948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:24.806374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:45513 (local address 127.21.214.126:40201)
0811 01:49:24.806889 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:24.806895 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:24.807879 (+ 984us) server_negotiation.cc:407] Connection header received
0811 01:49:24.809977 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:24.809982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:24.810107 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:24.810262 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:24.811965 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:24.812792 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:24.814710 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:24.815298 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:24.815460 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:24.819648 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:24.819658 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:24.819666 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:24.819721 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:24.823760 (+ 4039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:24.824762 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:24.824771 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:24.824777 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:24.824939 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:24.825514 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:24.825521 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:24.825527 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:24.825939 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:24.826101 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:24.827110 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0811 01:49:24.827409 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":287,"threads_started":1}
I20260811 01:49:24.829322 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:24.806853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40201 (local address 127.21.214.65:45513)
0811 01:49:24.807634 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:24.807676 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:24.809639 (+ 1963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:24.810525 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:24.810534 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:24.811093 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:49:24.811728 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:24.811783 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:24.812974 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:24.812979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:24.814542 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:24.814555 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:24.815574 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:24.815584 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:24.815846 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:24.816628 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:49:24.816659 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:49:24.819346 (+ 2687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:49:24.823965 (+ 4619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:49:24.823975 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:49:24.823991 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:49:24.824595 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:49:24.825078 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:49:24.825081 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:49:24.825084 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:49:24.825383 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:49:24.826066 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:49:24.826073 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:49:24.826611 (+ 538us) client_negotiation.cc:769] Sending connection context
0811 01:49:24.826940 (+ 329us) client_negotiation.cc:240] Negotiation successful
0811 01:49:24.827464 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":307,"threads_started":1}
I20260811 01:49:24.831696 9945 heartbeater.cc:347] Connected to a master server at 127.21.214.126:40201
I20260811 01:49:24.832144 9945 heartbeater.cc:464] Registering TS with master...
I20260811 01:49:24.833133 9945 heartbeater.cc:511] Master 127.21.214.126:40201 requested a full tablet report, sending...
I20260811 01:49:24.836082 9730 ts_manager.cc:194] Registered new tserver with Master: e33ad49a142e434cbeb5ccdfdb4aeada (127.21.214.65:38773)
I20260811 01:49:24.838059 9730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:45513
WARNING: no policy specified for kudu/127.21.214.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
I20260811 01:49:24.889963 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40201
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:37865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:49:25.263857 9953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:25.264227 9953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:25.264400 9953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:25.272389 9953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:25.272562 9953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:25.272749 9953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:49:25.283380 9953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:37865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40201
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:25.285033 9953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:25.286657 9953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:25.302035 9959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:25.302002 9961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:25.301961 9958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:25.302891 9953 server_base.cc:1034] running on GCE node
I20260811 01:49:25.303560 9953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:25.304790 9953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:25.306030 9953 hybrid_clock.cc:648] HybridClock initialized: now 1786412965305934 us; error 98 us; skew 500 ppm
I20260811 01:49:25.306396 9953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:25 dist-test-slave-9gcw krb5kdc[9676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412965, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:25.310140 9953 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:49:25.312028 9953 webserver.cc:460] Webserver started at http://127.21.214.66:34871/ using document root <none> and password file <none>
I20260811 01:49:25.312860 9953 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:25.313019 9953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:25.313421 9953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:25.316731 9953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "4edce26b8cb54024a030d03843596bf9"
format_stamp: "Formatted at 2026-08-11 01:49:25 on dist-test-slave-9gcw"
server_key: "442f9cb69ab27f78e1b9c23b58977f6a"
server_key_iv: "c0106b524919b35cecd8ea4fe250f261"
server_key_version: "encryptionkey@0"
I20260811 01:49:25.317647 9953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "4edce26b8cb54024a030d03843596bf9"
format_stamp: "Formatted at 2026-08-11 01:49:25 on dist-test-slave-9gcw"
server_key: "442f9cb69ab27f78e1b9c23b58977f6a"
server_key_iv: "c0106b524919b35cecd8ea4fe250f261"
server_key_version: "encryptionkey@0"
I20260811 01:49:25.323559 9953 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 01:49:25.327558 9970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:25.329118 9953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 01:49:25.329337 9953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "4edce26b8cb54024a030d03843596bf9"
format_stamp: "Formatted at 2026-08-11 01:49:25 on dist-test-slave-9gcw"
server_key: "442f9cb69ab27f78e1b9c23b58977f6a"
server_key_iv: "c0106b524919b35cecd8ea4fe250f261"
server_key_version: "encryptionkey@0"
I20260811 01:49:25.329563 9953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:25.344985 9953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:25.349815 9953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:25.350142 9953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:25.351222 9953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:25.352734 9953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:25.352860 9953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:25.353020 9953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:25.353101 9953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:25.391113 9953 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:38615
I20260811 01:49:25.391227 10098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:38615 every 8 connection(s)
I20260811 01:49:25.392544 9953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:49:25.394312 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 9953
I20260811 01:49:25.394526 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:49:25.395025 22361 external_mini_cluster.cc:1428] Setting key 6e05b69cb0985552cb93e81172bd5540
Aug 11 01:49:25 dist-test-slave-9gcw krb5kdc[9676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412965, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:49:25.415725 10101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:25.395751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:60829 (local address 127.21.214.126:40201)
0811 01:49:25.397663 (+ 1912us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:25.397672 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:25.397718 (+ 46us) server_negotiation.cc:407] Connection header received
0811 01:49:25.399912 (+ 2194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:25.399917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:25.400027 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:25.400154 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:25.401946 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:25.402740 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:25.404403 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:25.404899 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:25.405051 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:25.409064 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:25.409069 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:25.409072 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:25.409109 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:25.412022 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:25.412856 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:25.412860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:25.412862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:25.412999 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:25.413521 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:25.413525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:25.413527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:25.413822 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:25.414091 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:25.415232 (+ 1141us) server_negotiation.cc:299] Negotiation successful
0811 01:49:25.415447 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1660,"thread_start_us":1534,"threads_started":1}
I20260811 01:49:25.417380 10102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:25.396487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40201 (local address 127.21.214.66:60829)
0811 01:49:25.397547 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:25.397615 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:25.399599 (+ 1984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:25.400444 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:25.400454 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:25.401023 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:49:25.401749 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:25.401770 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:25.402919 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:25.402925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:25.404240 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:25.404253 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:25.405183 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:25.405192 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:25.405461 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:25.406219 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:49:25.406244 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:49:25.408789 (+ 2545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:49:25.412195 (+ 3406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:49:25.412205 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:49:25.412225 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:49:25.412694 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:49:25.413124 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:49:25.413129 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:49:25.413133 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:49:25.413381 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:49:25.414102 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:49:25.414113 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:49:25.414627 (+ 514us) client_negotiation.cc:769] Sending connection context
0811 01:49:25.414983 (+ 356us) client_negotiation.cc:240] Negotiation successful
0811 01:49:25.415698 (+ 715us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":800,"threads_started":1}
I20260811 01:49:25.419133 10099 heartbeater.cc:347] Connected to a master server at 127.21.214.126:40201
I20260811 01:49:25.419502 10099 heartbeater.cc:464] Registering TS with master...
I20260811 01:49:25.420303 10099 heartbeater.cc:511] Master 127.21.214.126:40201 requested a full tablet report, sending...
I20260811 01:49:25.422020 9730 ts_manager.cc:194] Registered new tserver with Master: 4edce26b8cb54024a030d03843596bf9 (127.21.214.66:38615)
I20260811 01:49:25.422885 9730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:60829
WARNING: no policy specified for kudu/127.21.214.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
I20260811 01:49:25.471135 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40201
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:37865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:49:25.841862 9945 heartbeater.cc:503] Master 127.21.214.126:40201 was elected leader, sending a full tablet report...
W20260811 01:49:25.852707 10106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:25.853075 10106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:25.853260 10106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:25.860934 10106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:25.861128 10106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:25.861347 10106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:49:25.872638 10106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:37865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:40201
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:25.874120 10106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:25.875797 10106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:25.889703 10113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:25.889590 10114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:25.889660 10116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:25.891093 10106 server_base.cc:1034] running on GCE node
I20260811 01:49:25.891779 10106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:25.892791 10106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:25.894050 10106 hybrid_clock.cc:648] HybridClock initialized: now 1786412965893999 us; error 40 us; skew 500 ppm
I20260811 01:49:25.894368 10106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:25 dist-test-slave-9gcw krb5kdc[9676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412965, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:25.898260 10106 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:49:25.899981 10106 webserver.cc:460] Webserver started at http://127.21.214.67:34215/ using document root <none> and password file <none>
I20260811 01:49:25.900772 10106 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:25.900933 10106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:25.901396 10106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:25.904649 10106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "7b01cea7d18b4af68522621b7af1b24f"
format_stamp: "Formatted at 2026-08-11 01:49:25 on dist-test-slave-9gcw"
server_key: "028bc596539dc3524ba56089b04f4562"
server_key_iv: "bf250335cb07cd92691da6deae52a46f"
server_key_version: "encryptionkey@0"
I20260811 01:49:25.905545 10106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "7b01cea7d18b4af68522621b7af1b24f"
format_stamp: "Formatted at 2026-08-11 01:49:25 on dist-test-slave-9gcw"
server_key: "028bc596539dc3524ba56089b04f4562"
server_key_iv: "bf250335cb07cd92691da6deae52a46f"
server_key_version: "encryptionkey@0"
I20260811 01:49:25.911144 10106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 01:49:25.914909 10125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:25.916370 10106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 01:49:25.916549 10106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "7b01cea7d18b4af68522621b7af1b24f"
format_stamp: "Formatted at 2026-08-11 01:49:25 on dist-test-slave-9gcw"
server_key: "028bc596539dc3524ba56089b04f4562"
server_key_iv: "bf250335cb07cd92691da6deae52a46f"
server_key_version: "encryptionkey@0"
I20260811 01:49:25.916818 10106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:25.933763 10106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:25.938761 10106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:25.939092 10106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:25.940155 10106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:25.941700 10106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:25.941809 10106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:25.941964 10106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:25.942044 10106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:25.978799 10106 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:46737
I20260811 01:49:25.978899 10252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:46737 every 8 connection(s)
I20260811 01:49:25.980249 10106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:49:25.985996 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 10106
I20260811 01:49:25.986251 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:49:25.986732 22361 external_mini_cluster.cc:1428] Setting key 28a1efbc79b7e978618f4aa39a656f48
Aug 11 01:49:25 dist-test-slave-9gcw krb5kdc[9676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412965, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:49:26.002300 10255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:25.983088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:53615 (local address 127.21.214.126:40201)
0811 01:49:25.983558 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:25.983563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:25.984866 (+ 1303us) server_negotiation.cc:407] Connection header received
0811 01:49:25.987294 (+ 2428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:25.987300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:25.987424 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:25.987553 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:25.989247 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:25.990088 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:25.991704 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:25.992137 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:25.992280 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:25.995956 (+ 3676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:25.995962 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:25.995965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:25.995999 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:25.998825 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:25.999615 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:25.999620 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:25.999622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:25.999749 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:26.000237 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:26.000241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:26.000243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:26.000527 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:26.000723 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:26.001571 (+ 848us) server_negotiation.cc:299] Negotiation successful
0811 01:49:26.001866 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":231,"threads_started":1}
I20260811 01:49:26.003394 10256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:25.983679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:40201 (local address 127.21.214.67:53615)
0811 01:49:25.984690 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:25.984747 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:25.986962 (+ 2215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:25.987840 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:25.987858 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:25.988420 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:49:25.989016 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:25.989034 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:25.990256 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:25.990262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:25.991546 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:25.991559 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:25.992381 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:25.992389 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:25.992639 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:25.993389 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:49:25.993413 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:49:25.995714 (+ 2301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:49:25.998983 (+ 3269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:49:25.998994 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:49:25.999013 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:49:25.999463 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:49:25.999866 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:49:25.999871 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:49:25.999875 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:49:26.000095 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:49:26.000695 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:49:26.000704 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:49:26.001095 (+ 391us) client_negotiation.cc:769] Sending connection context
0811 01:49:26.001456 (+ 361us) client_negotiation.cc:240] Negotiation successful
0811 01:49:26.002069 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":410,"threads_started":1}
I20260811 01:49:26.004943 10253 heartbeater.cc:347] Connected to a master server at 127.21.214.126:40201
I20260811 01:49:26.005357 10253 heartbeater.cc:464] Registering TS with master...
I20260811 01:49:26.006158 10253 heartbeater.cc:511] Master 127.21.214.126:40201 requested a full tablet report, sending...
I20260811 01:49:26.007649 9730 ts_manager.cc:194] Registered new tserver with Master: 7b01cea7d18b4af68522621b7af1b24f (127.21.214.67:46737)
I20260811 01:49:26.008744 9730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:53615
I20260811 01:49:26.015754 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:49:26.035233 10255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:26.020913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50682 (local address 127.21.214.126:40201)
0811 01:49:26.021194 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:26.021203 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:26.021585 (+ 382us) server_negotiation.cc:407] Connection header received
0811 01:49:26.021842 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:26.021846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:26.021967 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:26.022074 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:26.023765 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.024516 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.026001 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.026462 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.026606 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:26.029226 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:26.029232 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:26.029236 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:26.029271 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:26.032032 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:26.032774 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:26.032781 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:26.032839 (+ 58us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:26.032970 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:26.033448 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:26.033454 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:26.033459 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:26.033934 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:26.034196 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:26.034619 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 01:49:26.034886 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 01:49:26.040659 9730 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50682:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:49:26.048202 9730 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:49:26.086304 10273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:26.063141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58222 (local address 127.21.214.65:38773)
0811 01:49:26.076581 (+ 13440us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:26.076595 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:26.076637 (+ 42us) server_negotiation.cc:407] Connection header received
0811 01:49:26.076789 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:26.076802 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:26.077231 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:26.077515 (+ 284us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:26.078354 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.080135 (+ 1781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.083358 (+ 3223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.084938 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.085106 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:26.085454 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:26.085775 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 01:49:26.085945 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13265,"thread_start_us":13116,"threads_started":1}
I20260811 01:49:26.086355 10270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:26.062365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:38773 (local address 127.0.0.1:58222)
0811 01:49:26.075004 (+ 12639us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:26.075029 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:26.075176 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:26.077564 (+ 2388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:26.077576 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:26.077625 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:26.078091 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:26.078104 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.080334 (+ 2230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.080339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:26.083119 (+ 2780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:26.083137 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.085217 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.085225 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:26.085443 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:26.085506 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 01:49:26.085671 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 01:49:26.085914 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12465,"mutex_wait_us":64,"thread_start_us":13186,"threads_started":1}
I20260811 01:49:26.088155 10268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:26.063139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:38615 (local address 127.0.0.1:35744)
0811 01:49:26.075804 (+ 12665us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:26.075825 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:26.075944 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:26.080535 (+ 4591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:26.080543 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:26.080579 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:26.081080 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:26.081090 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.082596 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.082601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:26.085096 (+ 2495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:26.085115 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.087205 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.087212 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:26.087427 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:26.087476 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 01:49:26.087541 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 01:49:26.087686 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12514,"mutex_wait_us":28,"thread_start_us":10533,"threads_started":1}
I20260811 01:49:26.088289 10272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:26.062358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:46737 (local address 127.0.0.1:33588)
0811 01:49:26.072992 (+ 10634us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:26.073066 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:26.073401 (+ 335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:26.080161 (+ 6760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:26.080167 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:26.080199 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:26.080772 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:26.080794 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.082856 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.082862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:26.085137 (+ 2275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:26.085175 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.087494 (+ 2319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.087500 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:26.087667 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:26.087697 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 01:49:26.087754 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 01:49:26.087889 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10205,"mutex_wait_us":6,"thread_start_us":12218,"threads_started":1}
I20260811 01:49:26.089094 10271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:26.062462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33588 (local address 127.21.214.67:46737)
0811 01:49:26.079686 (+ 17224us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:26.079692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:26.079719 (+ 27us) server_negotiation.cc:407] Connection header received
0811 01:49:26.079800 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:26.079805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:26.079968 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:26.080128 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:26.081029 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.082701 (+ 1672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.085344 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.087253 (+ 1909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.088240 (+ 987us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:26.088492 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:26.088567 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 01:49:26.088744 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17059,"thread_start_us":16910,"threads_started":1}
I20260811 01:49:26.089568 10274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:26.063699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35744 (local address 127.21.214.66:38615)
0811 01:49:26.078418 (+ 14719us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:26.078432 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:26.078470 (+ 38us) server_negotiation.cc:407] Connection header received
0811 01:49:26.078604 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:26.078615 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:26.078948 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:26.080539 (+ 1591us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:26.081273 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.082420 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.085345 (+ 2925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.086944 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.088745 (+ 1801us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:26.088958 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:26.089026 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 01:49:26.089211 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14497,"thread_start_us":14309,"threads_started":1}
I20260811 01:49:26.091336 9870 tablet_service.cc:1467] Processing CreateTablet for tablet a46fe83798c147939f5052ab87ac20e4 (DEFAULT_TABLE table=test-table [id=c9872f29415f4d9d846a67db840e8165]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:49:26.093122 9870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a46fe83798c147939f5052ab87ac20e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:26.092680 10179 tablet_service.cc:1467] Processing CreateTablet for tablet a46fe83798c147939f5052ab87ac20e4 (DEFAULT_TABLE table=test-table [id=c9872f29415f4d9d846a67db840e8165]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:49:26.093108 10026 tablet_service.cc:1467] Processing CreateTablet for tablet a46fe83798c147939f5052ab87ac20e4 (DEFAULT_TABLE table=test-table [id=c9872f29415f4d9d846a67db840e8165]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:49:26.094133 10179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a46fe83798c147939f5052ab87ac20e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:26.094485 10026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a46fe83798c147939f5052ab87ac20e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:26.107448 10275 tablet_bootstrap.cc:492] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f: Bootstrap starting.
I20260811 01:49:26.109625 10276 tablet_bootstrap.cc:492] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada: Bootstrap starting.
I20260811 01:49:26.111616 10275 tablet_bootstrap.cc:654] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:26.113236 10275 log.cc:824] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:26.113577 10276 tablet_bootstrap.cc:654] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:26.115052 10276 log.cc:824] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:26.118875 10277 tablet_bootstrap.cc:492] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9: Bootstrap starting.
I20260811 01:49:26.124447 10275 tablet_bootstrap.cc:492] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f: No bootstrap required, opened a new log
I20260811 01:49:26.124662 10276 tablet_bootstrap.cc:492] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada: No bootstrap required, opened a new log
I20260811 01:49:26.124843 10275 ts_tablet_manager.cc:1397] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.008s
I20260811 01:49:26.124953 10276 ts_tablet_manager.cc:1397] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.006s
I20260811 01:49:26.127173 10277 tablet_bootstrap.cc:654] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:26.128744 10277 log.cc:824] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:26.136000 10276 raft_consensus.cc:348] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b01cea7d18b4af68522621b7af1b24f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 46737 } } peers { permanent_uuid: "e33ad49a142e434cbeb5ccdfdb4aeada" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38773 } } peers { permanent_uuid: "4edce26b8cb54024a030d03843596bf9" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38615 } }
I20260811 01:49:26.136453 10276 raft_consensus.cc:374] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:26.136605 10276 raft_consensus.cc:729] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e33ad49a142e434cbeb5ccdfdb4aeada, State: Initialized, Role: FOLLOWER
I20260811 01:49:26.137384 10276 consensus_queue.cc:260] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b01cea7d18b4af68522621b7af1b24f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 46737 } } peers { permanent_uuid: "e33ad49a142e434cbeb5ccdfdb4aeada" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38773 } } peers { permanent_uuid: "4edce26b8cb54024a030d03843596bf9" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38615 } }
I20260811 01:49:26.137557 10277 tablet_bootstrap.cc:492] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9: No bootstrap required, opened a new log
I20260811 01:49:26.137885 10277 ts_tablet_manager.cc:1397] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9: Time spent bootstrapping tablet: real 0.021s user 0.015s sys 0.000s
I20260811 01:49:26.139926 10275 raft_consensus.cc:348] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b01cea7d18b4af68522621b7af1b24f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 46737 } } peers { permanent_uuid: "e33ad49a142e434cbeb5ccdfdb4aeada" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38773 } } peers { permanent_uuid: "4edce26b8cb54024a030d03843596bf9" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38615 } }
I20260811 01:49:26.141268 10275 raft_consensus.cc:374] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:26.141393 10275 raft_consensus.cc:729] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b01cea7d18b4af68522621b7af1b24f, State: Initialized, Role: FOLLOWER
I20260811 01:49:26.142131 10275 consensus_queue.cc:260] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b01cea7d18b4af68522621b7af1b24f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 46737 } } peers { permanent_uuid: "e33ad49a142e434cbeb5ccdfdb4aeada" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38773 } } peers { permanent_uuid: "4edce26b8cb54024a030d03843596bf9" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38615 } }
I20260811 01:49:26.143570 10253 heartbeater.cc:503] Master 127.21.214.126:40201 was elected leader, sending a full tablet report...
I20260811 01:49:26.148761 10275 ts_tablet_manager.cc:1428] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f: Time spent starting tablet: real 0.024s user 0.014s sys 0.004s
I20260811 01:49:26.149400 10277 raft_consensus.cc:348] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b01cea7d18b4af68522621b7af1b24f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 46737 } } peers { permanent_uuid: "e33ad49a142e434cbeb5ccdfdb4aeada" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38773 } } peers { permanent_uuid: "4edce26b8cb54024a030d03843596bf9" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38615 } }
I20260811 01:49:26.149951 10277 raft_consensus.cc:374] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:26.150074 10277 raft_consensus.cc:729] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4edce26b8cb54024a030d03843596bf9, State: Initialized, Role: FOLLOWER
I20260811 01:49:26.150550 10276 ts_tablet_manager.cc:1428] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada: Time spent starting tablet: real 0.025s user 0.015s sys 0.011s
I20260811 01:49:26.150910 10277 consensus_queue.cc:260] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b01cea7d18b4af68522621b7af1b24f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 46737 } } peers { permanent_uuid: "e33ad49a142e434cbeb5ccdfdb4aeada" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38773 } } peers { permanent_uuid: "4edce26b8cb54024a030d03843596bf9" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38615 } }
I20260811 01:49:26.153082 10099 heartbeater.cc:503] Master 127.21.214.126:40201 was elected leader, sending a full tablet report...
I20260811 01:49:26.153736 10277 ts_tablet_manager.cc:1428] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9: Time spent starting tablet: real 0.016s user 0.004s sys 0.011s
I20260811 01:49:26.177043 10282 raft_consensus.cc:482] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:49:26.177448 10282 raft_consensus.cc:504] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b01cea7d18b4af68522621b7af1b24f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 46737 } } peers { permanent_uuid: "e33ad49a142e434cbeb5ccdfdb4aeada" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38773 } } peers { permanent_uuid: "4edce26b8cb54024a030d03843596bf9" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38615 } }
I20260811 01:49:26.179303 10282 leader_election.cc:288] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b01cea7d18b4af68522621b7af1b24f (127.21.214.67:46737), 4edce26b8cb54024a030d03843596bf9 (127.21.214.66:38615)
I20260811 01:49:26.189680 10271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:26.179874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:59623 (local address 127.21.214.67:46737)
0811 01:49:26.180085 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:26.180091 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:26.180901 (+ 810us) server_negotiation.cc:407] Connection header received
0811 01:49:26.181083 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:26.181089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:26.181222 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:26.181330 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:26.182433 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.183733 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.186980 (+ 3247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.188425 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.188608 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:26.188698 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:26.189227 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 01:49:26.189382 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 01:49:26.189764 10274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:26.179926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:37573 (local address 127.21.214.66:38615)
0811 01:49:26.180112 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:26.180118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:26.180886 (+ 768us) server_negotiation.cc:407] Connection header received
0811 01:49:26.181099 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:26.181105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:26.181214 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:26.181330 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:26.182433 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.183762 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.186980 (+ 3218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.188412 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.188608 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:26.188796 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:26.189227 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 01:49:26.189382 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 01:49:26.190045 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:26.179652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:46737 (local address 127.21.214.65:59623)
0811 01:49:26.180711 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:26.180738 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:26.180918 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:26.181369 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:26.181375 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:26.181435 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:26.182136 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:26.182157 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.183908 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.183913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:26.186765 (+ 2852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:26.186785 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.188718 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.188723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:26.188981 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:26.189055 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 01:49:26.189191 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 01:49:26.189409 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"mutex_wait_us":356,"spinlock_wait_cycles":76032,"thread_start_us":650,"threads_started":1}
I20260811 01:49:26.190276 10287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:26.179927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:38615 (local address 127.21.214.65:37573)
0811 01:49:26.180751 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:26.180769 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:26.180912 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:26.181369 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:26.181375 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:26.181435 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:26.182182 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:26.182192 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.183907 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.183913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:26.186765 (+ 2852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:26.186787 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.188705 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.188712 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:26.188999 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:26.189063 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 01:49:26.189191 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 01:49:26.189409 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":370,"spinlock_wait_cycles":21248,"thread_start_us":487,"threads_started":1}
I20260811 01:49:26.191362 10048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a46fe83798c147939f5052ab87ac20e4" candidate_uuid: "e33ad49a142e434cbeb5ccdfdb4aeada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4edce26b8cb54024a030d03843596bf9" is_pre_election: true
I20260811 01:49:26.191357 10202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a46fe83798c147939f5052ab87ac20e4" candidate_uuid: "e33ad49a142e434cbeb5ccdfdb4aeada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b01cea7d18b4af68522621b7af1b24f" is_pre_election: true
I20260811 01:49:26.191892 10202 raft_consensus.cc:2454] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e33ad49a142e434cbeb5ccdfdb4aeada in term 0.
I20260811 01:49:26.191890 10048 raft_consensus.cc:2454] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e33ad49a142e434cbeb5ccdfdb4aeada in term 0.
I20260811 01:49:26.192634 9817 leader_election.cc:302] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4edce26b8cb54024a030d03843596bf9, 7b01cea7d18b4af68522621b7af1b24f, e33ad49a142e434cbeb5ccdfdb4aeada; no voters:
I20260811 01:49:26.193071 10282 raft_consensus.cc:2781] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:49:26.193301 10282 raft_consensus.cc:482] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:49:26.193459 10282 raft_consensus.cc:3037] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:26.196029 10282 raft_consensus.cc:504] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b01cea7d18b4af68522621b7af1b24f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 46737 } } peers { permanent_uuid: "e33ad49a142e434cbeb5ccdfdb4aeada" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38773 } } peers { permanent_uuid: "4edce26b8cb54024a030d03843596bf9" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38615 } }
I20260811 01:49:26.196782 10282 leader_election.cc:288] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [CANDIDATE]: Term 1 election: Requested vote from peers 7b01cea7d18b4af68522621b7af1b24f (127.21.214.67:46737), 4edce26b8cb54024a030d03843596bf9 (127.21.214.66:38615)
I20260811 01:49:26.197319 10202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a46fe83798c147939f5052ab87ac20e4" candidate_uuid: "e33ad49a142e434cbeb5ccdfdb4aeada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b01cea7d18b4af68522621b7af1b24f"
I20260811 01:49:26.197371 10048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a46fe83798c147939f5052ab87ac20e4" candidate_uuid: "e33ad49a142e434cbeb5ccdfdb4aeada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4edce26b8cb54024a030d03843596bf9"
I20260811 01:49:26.197559 10202 raft_consensus.cc:3037] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:26.197629 10048 raft_consensus.cc:3037] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:26.200088 10202 raft_consensus.cc:2454] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e33ad49a142e434cbeb5ccdfdb4aeada in term 1.
I20260811 01:49:26.200244 10048 raft_consensus.cc:2454] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e33ad49a142e434cbeb5ccdfdb4aeada in term 1.
I20260811 01:49:26.200532 9817 leader_election.cc:302] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b01cea7d18b4af68522621b7af1b24f, e33ad49a142e434cbeb5ccdfdb4aeada; no voters:
I20260811 01:49:26.200873 10282 raft_consensus.cc:2781] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:49:26.201502 10282 raft_consensus.cc:686] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [term 1 LEADER]: Becoming Leader. State: Replica: e33ad49a142e434cbeb5ccdfdb4aeada, State: Running, Role: LEADER
I20260811 01:49:26.202075 10282 consensus_queue.cc:237] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b01cea7d18b4af68522621b7af1b24f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 46737 } } peers { permanent_uuid: "e33ad49a142e434cbeb5ccdfdb4aeada" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38773 } } peers { permanent_uuid: "4edce26b8cb54024a030d03843596bf9" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38615 } }
I20260811 01:49:26.207319 9730 catalog_manager.cc:5495] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada reported cstate change: term changed from 0 to 1, leader changed from <none> to e33ad49a142e434cbeb5ccdfdb4aeada (127.21.214.65). New cstate: current_term: 1 leader_uuid: "e33ad49a142e434cbeb5ccdfdb4aeada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b01cea7d18b4af68522621b7af1b24f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e33ad49a142e434cbeb5ccdfdb4aeada" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4edce26b8cb54024a030d03843596bf9" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38615 } health_report { overall_health: UNKNOWN } } }
W20260811 01:49:26.232126 10254 tablet.cc:2369] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:49:26.245908 10273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:26.240014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58238 (local address 127.21.214.65:38773)
0811 01:49:26.240211 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:26.240216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:26.240341 (+ 125us) server_negotiation.cc:407] Connection header received
0811 01:49:26.240557 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:26.240561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:26.240661 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:26.240801 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:49:26.241357 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.242075 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.243626 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.244056 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.244160 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:26.244682 (+ 522us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:49:26.245345 (+ 663us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:49:26.245460 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:26.245612 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 01:49:26.245690 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 01:49:26.255728 10048 raft_consensus.cc:1261] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9 [term 1 FOLLOWER]: Refusing update from remote peer e33ad49a142e434cbeb5ccdfdb4aeada: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:49:26.255726 10202 raft_consensus.cc:1261] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f [term 1 FOLLOWER]: Refusing update from remote peer e33ad49a142e434cbeb5ccdfdb4aeada: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:49:26.257088 10289 consensus_queue.cc:1035] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4edce26b8cb54024a030d03843596bf9" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 38615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:49:26.257606 10282 consensus_queue.cc:1035] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b01cea7d18b4af68522621b7af1b24f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:49:26.272488 10294 mvcc.cc:205] Tried to move back new op lower bound from 7317147509770330112 to 7317147509568266240. Current Snapshot: MvccSnapshot[applied={T|T < 7317147509770330112}]
I20260811 01:49:26.287937 9729 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:49:26.288290 9729 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:49:26.292524 9729 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 23d3eb51c7c342199836510d06e09a73: Sending DeleteTablet for 3 replicas of tablet a46fe83798c147939f5052ab87ac20e4
I20260811 01:49:26.294127 10179 tablet_service.cc:1514] Processing DeleteTablet for tablet a46fe83798c147939f5052ab87ac20e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:26 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:33588
I20260811 01:49:26.294106 9870 tablet_service.cc:1514] Processing DeleteTablet for tablet a46fe83798c147939f5052ab87ac20e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:26 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:58222
I20260811 01:49:26.295068 10026 tablet_service.cc:1514] Processing DeleteTablet for tablet a46fe83798c147939f5052ab87ac20e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:26 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:35744
I20260811 01:49:26.295676 10304 tablet_replica.cc:330] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada: stopping tablet replica
I20260811 01:49:26.296238 10304 raft_consensus.cc:2229] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:49:26.296562 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 9798
I20260811 01:49:26.297994 10304 raft_consensus.cc:2258] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:49:26.299019 10303 tablet_replica.cc:330] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f: stopping tablet replica
I20260811 01:49:26.299634 10303 raft_consensus.cc:2229] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:49:26.300311 10303 raft_consensus.cc:2258] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:49:26.300784 10304 ts_tablet_manager.cc:1905] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:49:26.303460 10303 ts_tablet_manager.cc:1905] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:49:26.307878 10305 tablet_replica.cc:330] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9: stopping tablet replica
I20260811 01:49:26.308415 10305 raft_consensus.cc:2229] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:49:26.309005 10304 ts_tablet_manager.cc:1918] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:49:26.309114 10305 raft_consensus.cc:2258] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:49:26.309489 10304 log.cc:1196] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/a46fe83798c147939f5052ab87ac20e4
I20260811 01:49:26.310806 10304 ts_tablet_manager.cc:1939] T a46fe83798c147939f5052ab87ac20e4 P e33ad49a142e434cbeb5ccdfdb4aeada: Deleting consensus metadata
I20260811 01:49:26.312557 10305 ts_tablet_manager.cc:1905] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:49:26.315138 9715 catalog_manager.cc:4841] TS e33ad49a142e434cbeb5ccdfdb4aeada (127.21.214.65:38773): tablet a46fe83798c147939f5052ab87ac20e4 (table test-table [id=c9872f29415f4d9d846a67db840e8165]) successfully deleted
I20260811 01:49:26.316337 10303 ts_tablet_manager.cc:1918] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:49:26.316606 10303 log.cc:1196] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/a46fe83798c147939f5052ab87ac20e4
I20260811 01:49:26.317365 10303 ts_tablet_manager.cc:1939] T a46fe83798c147939f5052ab87ac20e4 P 7b01cea7d18b4af68522621b7af1b24f: Deleting consensus metadata
I20260811 01:49:26.319579 9712 catalog_manager.cc:4841] TS 7b01cea7d18b4af68522621b7af1b24f (127.21.214.67:46737): tablet a46fe83798c147939f5052ab87ac20e4 (table test-table [id=c9872f29415f4d9d846a67db840e8165]) successfully deleted
Aug 11 01:49:26 dist-test-slave-9gcw krb5kdc[9676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:49:26.322471 10305 ts_tablet_manager.cc:1918] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:49:26.322665 10305 log.cc:1196] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/a46fe83798c147939f5052ab87ac20e4
I20260811 01:49:26.323244 10305 ts_tablet_manager.cc:1939] T a46fe83798c147939f5052ab87ac20e4 P 4edce26b8cb54024a030d03843596bf9: Deleting consensus metadata
I20260811 01:49:26.325397 9714 catalog_manager.cc:4841] TS 4edce26b8cb54024a030d03843596bf9 (127.21.214.66:38615): tablet a46fe83798c147939f5052ab87ac20e4 (table test-table [id=c9872f29415f4d9d846a67db840e8165]) successfully deleted
I20260811 01:49:26.329339 10273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:26.300473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58242 (local address 127.21.214.65:38773)
0811 01:49:26.300698 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:26.300706 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:26.307514 (+ 6808us) server_negotiation.cc:407] Connection header received
0811 01:49:26.307985 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:26.307990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:26.308079 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:26.308154 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:26.311047 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.312467 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.314725 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.315688 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.315792 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:26.322068 (+ 6276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:26.322089 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:26.322105 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:26.322153 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:26.325994 (+ 3841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:26.326949 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:26.326957 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:26.326960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:26.327128 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:26.327582 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:26.327586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:26.327588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:26.327985 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:26.328346 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:26.328642 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 01:49:26.328893 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 01:49:26.330075 9938 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:26.432070 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 9798
I20260811 01:49:26.458216 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 9953
Aug 11 01:49:26 dist-test-slave-9gcw krb5kdc[9676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.66@KRBTEST.COM
I20260811 01:49:26.476218 10274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:26.459235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35756 (local address 127.21.214.66:38615)
0811 01:49:26.459437 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:26.459442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:26.459464 (+ 22us) server_negotiation.cc:407] Connection header received
0811 01:49:26.459693 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:26.459698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:26.459766 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:26.459870 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:26.461299 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.462132 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.463709 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.464192 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.464305 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:26.469603 (+ 5298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:26.469628 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:26.469653 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:26.469709 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:26.473006 (+ 3297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:26.473940 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:26.473949 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:26.473951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:26.474106 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:26.474540 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:26.474543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:26.474545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:26.474969 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:26.475243 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:26.475693 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 01:49:26.475900 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 01:49:26.477063 10093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:26.570359 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 9953
I20260811 01:49:26.594027 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 10106
Aug 11 01:49:26 dist-test-slave-9gcw krb5kdc[9676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.67@KRBTEST.COM
I20260811 01:49:26.611222 10271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:26.594898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33598 (local address 127.21.214.67:46737)
0811 01:49:26.595088 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:26.595093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:26.595118 (+ 25us) server_negotiation.cc:407] Connection header received
0811 01:49:26.595294 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:26.595298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:26.595381 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:26.595473 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:26.596854 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.597880 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.599169 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:26.599563 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:26.599660 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:26.604524 (+ 4864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:26.604545 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:26.604566 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:26.604707 (+ 141us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:26.608045 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:26.608856 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:26.608865 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:26.608867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:26.609025 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:26.609456 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:26.609460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:26.609462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:26.609944 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:26.610186 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:26.610685 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 01:49:26.610905 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 01:49:26.612037 10247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:26.705829 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 10106
I20260811 01:49:26.728595 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 9691
I20260811 01:49:26.729957 9763 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:26.848757 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 9691
2026-08-11T01:49:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4867 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T01:49:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:49:26Z Disabled control of system clock
I20260811 01:49:26.955058 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:34097
--webserver_interface=127.21.214.126
--webserver_port=0
--builtin_ntp_servers=127.21.214.84:32875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:34097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 01:49:27.324577 10314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:27.324950 10314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:27.325100 10314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:27.332934 10314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:49:27.333066 10314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:27.333108 10314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:27.333276 10314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:49:27.333334 10314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:49:27.343261 10314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:32875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:34097
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:34097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:27.344724 10314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:27.347261 10314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:27.361091 10322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:27.361246 10326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:27.361632 10323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:27.362246 10314 server_base.cc:1034] running on GCE node
I20260811 01:49:27.363189 10314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:27.364894 10314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:27.366191 10314 hybrid_clock.cc:648] HybridClock initialized: now 1786412967366141 us; error 72 us; skew 500 ppm
I20260811 01:49:27.366612 10314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:49:27.374862 10314 webserver.cc:460] Webserver started at http://127.21.214.126:43315/ using document root <none> and password file <none>
I20260811 01:49:27.375778 10314 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:27.375952 10314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:27.376368 10314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:27.379899 10314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "e06a86eeed9e45f68a324faac293521c"
format_stamp: "Formatted at 2026-08-11 01:49:27 on dist-test-slave-9gcw"
server_key: "11afc67313b27f23d9e01f5a6f0e7b6b"
server_key_iv: "4de232cd3cc55ef0be6f7f8c67a79fcf"
server_key_version: "encryptionkey@0"
I20260811 01:49:27.380918 10314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "e06a86eeed9e45f68a324faac293521c"
format_stamp: "Formatted at 2026-08-11 01:49:27 on dist-test-slave-9gcw"
server_key: "11afc67313b27f23d9e01f5a6f0e7b6b"
server_key_iv: "4de232cd3cc55ef0be6f7f8c67a79fcf"
server_key_version: "encryptionkey@0"
I20260811 01:49:27.387209 10314 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 01:49:27.391510 10333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:27.393114 10314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 01:49:27.393431 10314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "e06a86eeed9e45f68a324faac293521c"
format_stamp: "Formatted at 2026-08-11 01:49:27 on dist-test-slave-9gcw"
server_key: "11afc67313b27f23d9e01f5a6f0e7b6b"
server_key_iv: "4de232cd3cc55ef0be6f7f8c67a79fcf"
server_key_version: "encryptionkey@0"
I20260811 01:49:27.393708 10314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:27.416937 10314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:27.418049 10314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:27.418412 10314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:27.440433 10314 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:34097
I20260811 01:49:27.440460 10388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:34097 every 8 connection(s)
I20260811 01:49:27.442171 10314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:49:27.446461 10390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:27.449362 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 10314
I20260811 01:49:27.449548 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:49:27.450130 22361 external_mini_cluster.cc:1428] Setting key 3b85ec5939985509f3ca357045245141
I20260811 01:49:27.460692 10390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c: Bootstrap starting.
I20260811 01:49:27.464886 10390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:27.466241 10390 log.cc:824] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:27.466912 10394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:27.452884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47076 (local address 127.21.214.126:34097)
0811 01:49:27.454484 (+ 1600us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:27.454496 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:27.454545 (+ 49us) server_negotiation.cc:407] Connection header received
0811 01:49:27.456426 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:27.456462 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:27.457031 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:27.457524 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:27.458245 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:27.459685 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:27.461699 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:27.462454 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:27.462622 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:27.464009 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:27.464018 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:27.464033 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:27.464635 (+ 602us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:27.464759 (+ 124us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:27.464784 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:27.464901 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:27.465323 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 01:49:27.465750 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1116,"thread_start_us":788,"threads_started":1}
I20260811 01:49:27.470023 10390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c: No bootstrap required, opened a new log
I20260811 01:49:27.478596 10390 raft_consensus.cc:348] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06a86eeed9e45f68a324faac293521c" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34097 } }
I20260811 01:49:27.479056 10390 raft_consensus.cc:374] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:27.479164 10390 raft_consensus.cc:729] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06a86eeed9e45f68a324faac293521c, State: Initialized, Role: FOLLOWER
I20260811 01:49:27.479907 10390 consensus_queue.cc:260] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06a86eeed9e45f68a324faac293521c" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34097 } }
I20260811 01:49:27.480165 10390 raft_consensus.cc:388] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:49:27.480314 10390 raft_consensus.cc:482] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:49:27.480520 10390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:27.483378 10390 raft_consensus.cc:504] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06a86eeed9e45f68a324faac293521c" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34097 } }
I20260811 01:49:27.483901 10390 leader_election.cc:302] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e06a86eeed9e45f68a324faac293521c; no voters:
I20260811 01:49:27.484493 10390 leader_election.cc:288] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:49:27.484773 10396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:49:27.485656 10396 raft_consensus.cc:686] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c [term 1 LEADER]: Becoming Leader. State: Replica: e06a86eeed9e45f68a324faac293521c, State: Running, Role: LEADER
I20260811 01:49:27.486272 10396 consensus_queue.cc:237] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06a86eeed9e45f68a324faac293521c" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34097 } }
I20260811 01:49:27.487268 10390 sys_catalog.cc:564] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:49:27.489591 10398 sys_catalog.cc:455] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e06a86eeed9e45f68a324faac293521c. Latest consensus state: current_term: 1 leader_uuid: "e06a86eeed9e45f68a324faac293521c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06a86eeed9e45f68a324faac293521c" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34097 } } }
I20260811 01:49:27.489912 10397 sys_catalog.cc:455] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e06a86eeed9e45f68a324faac293521c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06a86eeed9e45f68a324faac293521c" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34097 } } }
I20260811 01:49:27.490207 10398 sys_catalog.cc:458] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:27.490229 10397 sys_catalog.cc:458] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:27.492429 10405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:49:27.498799 10405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:49:27.508442 10405 catalog_manager.cc:1347] Generated new cluster ID: d5e925b623314287b674420beee69df1
I20260811 01:49:27.508594 10405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:49:27.520272 10405 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:49:27.521934 10405 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:49:27.532747 10405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c: Generated new TSK 0
I20260811 01:49:27.533854 10405 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:49:27.543968 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:34097
--builtin_ntp_servers=127.21.214.84:32875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 01:49:27.920502 10419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:27.920899 10419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:27.921087 10419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:27.928829 10419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:27.929000 10419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:27.929209 10419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:49:27.939888 10419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:32875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:34097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:27.941386 10419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:27.943264 10419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:27.956959 10427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:27.956969 10426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:27.956982 10429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:27.958439 10419 server_base.cc:1034] running on GCE node
I20260811 01:49:27.959105 10419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:27.960109 10419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:27.961350 10419 hybrid_clock.cc:648] HybridClock initialized: now 1786412967961296 us; error 46 us; skew 500 ppm
I20260811 01:49:27.961705 10419 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:49:27.964088 10419 webserver.cc:460] Webserver started at http://127.21.214.65:36449/ using document root <none> and password file <none>
I20260811 01:49:27.964946 10419 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:27.965092 10419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:27.965461 10419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:27.968775 10419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "d924c3a3ae57461c98d9e3c4470824cb"
format_stamp: "Formatted at 2026-08-11 01:49:27 on dist-test-slave-9gcw"
server_key: "c05b857039312689a8188a33ab923c51"
server_key_iv: "af52c534fce20289f4992e6593b6a367"
server_key_version: "encryptionkey@0"
I20260811 01:49:27.969769 10419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "d924c3a3ae57461c98d9e3c4470824cb"
format_stamp: "Formatted at 2026-08-11 01:49:27 on dist-test-slave-9gcw"
server_key: "c05b857039312689a8188a33ab923c51"
server_key_iv: "af52c534fce20289f4992e6593b6a367"
server_key_version: "encryptionkey@0"
I20260811 01:49:27.975373 10419 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 01:49:27.979220 10435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:27.980762 10419 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 01:49:27.980911 10419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "d924c3a3ae57461c98d9e3c4470824cb"
format_stamp: "Formatted at 2026-08-11 01:49:27 on dist-test-slave-9gcw"
server_key: "c05b857039312689a8188a33ab923c51"
server_key_iv: "af52c534fce20289f4992e6593b6a367"
server_key_version: "encryptionkey@0"
I20260811 01:49:27.981112 10419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:28.003906 10419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:28.004839 10419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:28.005136 10419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:28.006191 10419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:28.007718 10419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:28.007807 10419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:28.007880 10419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:28.007936 10419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:28.043133 10419 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:36039
I20260811 01:49:28.043236 10564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:36039 every 8 connection(s)
I20260811 01:49:28.044574 10419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:49:28.049384 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 10419
I20260811 01:49:28.049603 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:49:28.050060 22361 external_mini_cluster.cc:1428] Setting key ea71af5a131b0ca38232a01981b8167b
I20260811 01:49:28.052951 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:34097
--builtin_ntp_servers=127.21.214.84:32875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 01:49:28.059463 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:28.047425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:41207 (local address 127.21.214.126:34097)
0811 01:49:28.047903 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:28.047908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:28.049475 (+ 1567us) server_negotiation.cc:407] Connection header received
0811 01:49:28.051697 (+ 2222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:28.051704 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:28.051807 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:28.051946 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:28.053396 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:28.054158 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:28.056021 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:28.056481 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:28.056653 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:28.058421 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:28.058427 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:28.058430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:28.058454 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:28.058500 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:28.058514 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:28.058625 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:28.058902 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 01:49:28.059019 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":236,"threads_started":1}
I20260811 01:49:28.060662 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:28.048068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:34097 (local address 127.21.214.65:41207)
0811 01:49:28.049292 (+ 1224us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:28.049336 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:28.051357 (+ 2021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:28.052230 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:28.052240 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:28.052360 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:49:28.053050 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:28.053070 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:28.054369 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:28.054375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:28.055829 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:28.055844 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:28.056776 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:28.056784 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:28.057073 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:28.057976 (+ 903us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:49:28.057978 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:49:28.058026 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:49:28.058161 (+ 135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:49:28.058632 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:49:28.058648 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:49:28.058665 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 01:49:28.058873 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 01:49:28.059358 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":316,"threads_started":1}
I20260811 01:49:28.062567 10565 heartbeater.cc:347] Connected to a master server at 127.21.214.126:34097
I20260811 01:49:28.062952 10565 heartbeater.cc:464] Registering TS with master...
I20260811 01:49:28.063798 10565 heartbeater.cc:511] Master 127.21.214.126:34097 requested a full tablet report, sending...
I20260811 01:49:28.066188 10350 ts_manager.cc:194] Registered new tserver with Master: d924c3a3ae57461c98d9e3c4470824cb (127.21.214.65:36039)
I20260811 01:49:28.067751 10350 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.214.65:41207
W20260811 01:49:28.426134 10569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:28.426515 10569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:28.426785 10569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:28.434986 10569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:28.435179 10569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:28.435410 10569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:49:28.449168 10569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:32875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:34097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:28.450933 10569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:28.453608 10569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:28.470970 10577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:28.471036 10580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:28.471268 10576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:28.471736 10569 server_base.cc:1034] running on GCE node
I20260811 01:49:28.472505 10569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:28.473827 10569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:28.475107 10569 hybrid_clock.cc:648] HybridClock initialized: now 1786412968475029 us; error 76 us; skew 500 ppm
I20260811 01:49:28.475502 10569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:49:28.478214 10569 webserver.cc:460] Webserver started at http://127.21.214.66:33093/ using document root <none> and password file <none>
I20260811 01:49:28.479018 10569 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:28.479163 10569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:28.479537 10569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:28.482784 10569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "e119c267fd9a48718501f760222b923d"
format_stamp: "Formatted at 2026-08-11 01:49:28 on dist-test-slave-9gcw"
server_key: "3140b075d3d3f9f93115641f0f0e7d50"
server_key_iv: "927ec2322527c150e2b8ebb3b9af8359"
server_key_version: "encryptionkey@0"
I20260811 01:49:28.483642 10569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "e119c267fd9a48718501f760222b923d"
format_stamp: "Formatted at 2026-08-11 01:49:28 on dist-test-slave-9gcw"
server_key: "3140b075d3d3f9f93115641f0f0e7d50"
server_key_iv: "927ec2322527c150e2b8ebb3b9af8359"
server_key_version: "encryptionkey@0"
I20260811 01:49:28.489735 10569 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 01:49:28.493649 10586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:28.495028 10569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:49:28.495225 10569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "e119c267fd9a48718501f760222b923d"
format_stamp: "Formatted at 2026-08-11 01:49:28 on dist-test-slave-9gcw"
server_key: "3140b075d3d3f9f93115641f0f0e7d50"
server_key_iv: "927ec2322527c150e2b8ebb3b9af8359"
server_key_version: "encryptionkey@0"
I20260811 01:49:28.495471 10569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:28.513085 10569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:28.514075 10569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:28.514410 10569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:28.515472 10569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:28.517264 10569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:28.517397 10569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:28.517560 10569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:28.517654 10569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:28.553735 10569 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:37421
I20260811 01:49:28.553828 10711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:37421 every 8 connection(s)
I20260811 01:49:28.555184 10569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:49:28.559139 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 10569
I20260811 01:49:28.559346 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:49:28.560045 22361 external_mini_cluster.cc:1428] Setting key 1b6a9a5ff9f9d3d31b3f4e352524577a
I20260811 01:49:28.563993 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:34097
--builtin_ntp_servers=127.21.214.84:32875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 01:49:28.571908 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:28.558096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:51691 (local address 127.21.214.126:34097)
0811 01:49:28.558288 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:28.558293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:28.560188 (+ 1895us) server_negotiation.cc:407] Connection header received
0811 01:49:28.562818 (+ 2630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:28.562822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:28.562888 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:28.562962 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:28.564652 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:28.565580 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:28.568342 (+ 2762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:28.568820 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:28.568934 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:28.570967 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:28.570976 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:28.570983 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:28.571015 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:28.571063 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:28.571082 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:28.571151 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:28.571533 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 01:49:28.571634 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 01:49:28.573709 10714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:28.558693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:34097 (local address 127.21.214.66:51691)
0811 01:49:28.559953 (+ 1260us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:28.560019 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:28.562474 (+ 2455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:28.563301 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:28.563315 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:28.563444 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:49:28.564424 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:28.564454 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:28.565759 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:28.565769 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:28.568133 (+ 2364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:28.568154 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:28.569091 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:28.569100 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:28.569534 (+ 434us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:28.570494 (+ 960us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:49:28.570500 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:49:28.570565 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:49:28.570693 (+ 128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:49:28.571266 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:49:28.571280 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:49:28.571292 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 01:49:28.571498 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 01:49:28.572046 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":352,"threads_started":1}
I20260811 01:49:28.575474 10712 heartbeater.cc:347] Connected to a master server at 127.21.214.126:34097
I20260811 01:49:28.575845 10712 heartbeater.cc:464] Registering TS with master...
I20260811 01:49:28.576982 10712 heartbeater.cc:511] Master 127.21.214.126:34097 requested a full tablet report, sending...
I20260811 01:49:28.578999 10350 ts_manager.cc:194] Registered new tserver with Master: e119c267fd9a48718501f760222b923d (127.21.214.66:37421)
I20260811 01:49:28.580339 10350 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.214.66:51691
W20260811 01:49:28.930964 10716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:28.931479 10716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:28.931694 10716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:28.943039 10716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:28.943255 10716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:28.943498 10716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:49:28.956682 10716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:32875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:34097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:28.958343 10716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:28.960172 10716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:28.975062 10722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:28.975036 10727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:28.975040 10723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:28.975870 10716 server_base.cc:1034] running on GCE node
I20260811 01:49:28.976843 10716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:28.978039 10716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:28.979254 10716 hybrid_clock.cc:648] HybridClock initialized: now 1786412968979203 us; error 45 us; skew 500 ppm
I20260811 01:49:28.979611 10716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:49:28.982224 10716 webserver.cc:460] Webserver started at http://127.21.214.67:41009/ using document root <none> and password file <none>
I20260811 01:49:28.983001 10716 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:28.983095 10716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:28.983415 10716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:28.986748 10716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "6391a80c14744499ac1146afac57af7d"
format_stamp: "Formatted at 2026-08-11 01:49:28 on dist-test-slave-9gcw"
server_key: "e684feecaf3946d213eb13356d04547c"
server_key_iv: "54acf1e8513b8b3b49f792525ab82c68"
server_key_version: "encryptionkey@0"
I20260811 01:49:28.987695 10716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "6391a80c14744499ac1146afac57af7d"
format_stamp: "Formatted at 2026-08-11 01:49:28 on dist-test-slave-9gcw"
server_key: "e684feecaf3946d213eb13356d04547c"
server_key_iv: "54acf1e8513b8b3b49f792525ab82c68"
server_key_version: "encryptionkey@0"
I20260811 01:49:28.994123 10716 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 01:49:28.998202 10735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:28.999938 10716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:49:29.000111 10716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "6391a80c14744499ac1146afac57af7d"
format_stamp: "Formatted at 2026-08-11 01:49:28 on dist-test-slave-9gcw"
server_key: "e684feecaf3946d213eb13356d04547c"
server_key_iv: "54acf1e8513b8b3b49f792525ab82c68"
server_key_version: "encryptionkey@0"
I20260811 01:49:29.000319 10716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:29.017742 10716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:29.018697 10716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:29.019007 10716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:29.020088 10716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:29.021946 10716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:29.022089 10716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:29.022262 10716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:29.022349 10716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:29.060344 10716 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:38783
I20260811 01:49:29.060403 10867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:38783 every 8 connection(s)
I20260811 01:49:29.061848 10716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:49:29.071622 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 10716
I20260811 01:49:29.071918 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:49:29.071887 10565 heartbeater.cc:503] Master 127.21.214.126:34097 was elected leader, sending a full tablet report...
I20260811 01:49:29.072543 22361 external_mini_cluster.cc:1428] Setting key ccaed4c685136cf839c1391f472e7e56
I20260811 01:49:29.076908 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:29.064615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:54005 (local address 127.21.214.126:34097)
0811 01:49:29.064855 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:29.064859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:29.066491 (+ 1632us) server_negotiation.cc:407] Connection header received
0811 01:49:29.068455 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:29.068460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:29.068529 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:29.068622 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:29.069921 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.070699 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.073295 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.074007 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.074166 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:29.076017 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:29.076024 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:29.076030 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:29.076055 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:29.076087 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:29.076099 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:29.076199 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:29.076447 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 01:49:29.076558 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260811 01:49:29.078095 10870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:29.065233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:34097 (local address 127.21.214.67:54005)
0811 01:49:29.066300 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:29.066342 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:29.068168 (+ 1826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:29.068868 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:29.068877 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:29.068984 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:49:29.069751 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:29.069771 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.070884 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.070888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:29.073077 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:29.073092 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.074291 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.074300 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:29.074864 (+ 564us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:29.075669 (+ 805us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:49:29.075673 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:49:29.075719 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:49:29.075807 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:49:29.076204 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:49:29.076219 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:49:29.076229 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 01:49:29.076421 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 01:49:29.076898 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":317,"threads_started":1}
I20260811 01:49:29.079530 10868 heartbeater.cc:347] Connected to a master server at 127.21.214.126:34097
I20260811 01:49:29.079953 10868 heartbeater.cc:464] Registering TS with master...
I20260811 01:49:29.080765 10868 heartbeater.cc:511] Master 127.21.214.126:34097 requested a full tablet report, sending...
I20260811 01:49:29.082260 10349 ts_manager.cc:194] Registered new tserver with Master: 6391a80c14744499ac1146afac57af7d (127.21.214.67:38783)
I20260811 01:49:29.083155 10349 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.214.67:54005
I20260811 01:49:29.088296 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:49:29.100040 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:29.092969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47092 (local address 127.21.214.126:34097)
0811 01:49:29.093220 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:29.093227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:29.093692 (+ 465us) server_negotiation.cc:407] Connection header received
0811 01:49:29.093856 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:29.093860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:29.093924 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:29.094015 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:29.094503 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.095759 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.097509 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.097956 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.098102 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:29.099214 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:29.099220 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:29.099225 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:29.099254 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:29.099293 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:29.099306 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:29.099405 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:29.099582 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 01:49:29.099698 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 01:49:29.104490 10349 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:49:29.111691 10349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:49:29.135426 10884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:29.124787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:38783 (local address 127.0.0.1:55172)
0811 01:49:29.125702 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:29.125751 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:29.125913 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:29.126625 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:29.126633 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:29.126664 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:29.127194 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:29.127209 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.129274 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.129292 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:29.132157 (+ 2865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:29.132176 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.134240 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.134248 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:29.134494 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:29.134563 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 01:49:29.134748 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 01:49:29.134991 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":3,"thread_start_us":498,"threads_started":1}
I20260811 01:49:29.135440 10885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:29.124940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55172 (local address 127.21.214.67:38783)
0811 01:49:29.125954 (+ 1014us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:29.125969 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:29.125999 (+ 30us) server_negotiation.cc:407] Connection header received
0811 01:49:29.126123 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:29.126130 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:29.126371 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:29.126610 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:29.127448 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.129103 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.132422 (+ 3319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.134013 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.134159 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:29.134493 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:29.134810 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 01:49:29.134996 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":850,"thread_start_us":693,"threads_started":1}
I20260811 01:49:29.138062 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:29.126109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:37421 (local address 127.0.0.1:38240)
0811 01:49:29.127331 (+ 1222us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:29.127351 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:29.127458 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:29.130587 (+ 3129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:29.130591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:29.130622 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:29.131059 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:29.131070 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.132844 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.132853 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:29.134650 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:29.134665 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.137221 (+ 2556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.137228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:29.137386 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:29.137418 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 01:49:29.137541 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 01:49:29.137686 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1072,"mutex_wait_us":44,"thread_start_us":918,"threads_started":1}
I20260811 01:49:29.138100 10887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:29.126461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38240 (local address 127.21.214.66:37421)
0811 01:49:29.130088 (+ 3627us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:29.130094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:29.130120 (+ 26us) server_negotiation.cc:407] Connection header received
0811 01:49:29.130207 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:29.130212 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:29.130387 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:29.130613 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:29.131246 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.132710 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.134891 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.136991 (+ 2100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.137141 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:29.137421 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:29.137594 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 01:49:29.137757 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3466,"thread_start_us":3338,"threads_started":1}
I20260811 01:49:29.138909 10889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:29.128346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:36039 (local address 127.0.0.1:41114)
0811 01:49:29.129722 (+ 1376us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:29.129739 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:29.129880 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:29.130894 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:29.130902 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:29.130930 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:29.131387 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:29.131401 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.133368 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.133375 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:29.135693 (+ 2318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:29.135706 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.138128 (+ 2422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.138135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:29.138314 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:29.138355 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 01:49:29.138460 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 01:49:29.138612 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1245,"mutex_wait_us":35,"thread_start_us":1078,"threads_started":1}
I20260811 01:49:29.139113 10888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:29.127980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41114 (local address 127.21.214.65:36039)
0811 01:49:29.129183 (+ 1203us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:29.129194 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:29.130025 (+ 831us) server_negotiation.cc:407] Connection header received
0811 01:49:29.130151 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:29.130159 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:29.130417 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:29.130635 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:29.131598 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.133189 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.135893 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.137595 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.137733 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:29.137983 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:29.138549 (+ 566us) server_negotiation.cc:299] Negotiation successful
0811 01:49:29.138737 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":985,"thread_start_us":844,"threads_started":1}
I20260811 01:49:29.140683 10787 tablet_service.cc:1467] Processing CreateTablet for tablet efe3f40a3b57484b8107f99c1be10d5d (DEFAULT_TABLE table=test-table [id=92fef493cc104513ba343bcec3cc94b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:49:29.142477 10787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efe3f40a3b57484b8107f99c1be10d5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:29.142491 10485 tablet_service.cc:1467] Processing CreateTablet for tablet efe3f40a3b57484b8107f99c1be10d5d (DEFAULT_TABLE table=test-table [id=92fef493cc104513ba343bcec3cc94b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:49:29.142477 10638 tablet_service.cc:1467] Processing CreateTablet for tablet efe3f40a3b57484b8107f99c1be10d5d (DEFAULT_TABLE table=test-table [id=92fef493cc104513ba343bcec3cc94b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:49:29.143780 10485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efe3f40a3b57484b8107f99c1be10d5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:29.143906 10638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efe3f40a3b57484b8107f99c1be10d5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:29.159559 10890 tablet_bootstrap.cc:492] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb: Bootstrap starting.
I20260811 01:49:29.164408 10893 tablet_bootstrap.cc:492] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d: Bootstrap starting.
I20260811 01:49:29.165297 10890 tablet_bootstrap.cc:654] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:29.165577 10891 tablet_bootstrap.cc:492] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d: Bootstrap starting.
I20260811 01:49:29.167691 10890 log.cc:824] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:29.169319 10891 tablet_bootstrap.cc:654] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:29.169329 10893 tablet_bootstrap.cc:654] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:29.170655 10891 log.cc:824] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:29.170653 10893 log.cc:824] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:29.172055 10890 tablet_bootstrap.cc:492] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb: No bootstrap required, opened a new log
I20260811 01:49:29.172540 10890 ts_tablet_manager.cc:1397] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.007s
I20260811 01:49:29.173652 10891 tablet_bootstrap.cc:492] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d: No bootstrap required, opened a new log
I20260811 01:49:29.173983 10891 ts_tablet_manager.cc:1397] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260811 01:49:29.174191 10893 tablet_bootstrap.cc:492] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d: No bootstrap required, opened a new log
I20260811 01:49:29.174492 10893 ts_tablet_manager.cc:1397] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.006s
I20260811 01:49:29.181761 10891 raft_consensus.cc:348] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6391a80c14744499ac1146afac57af7d" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38783 } } peers { permanent_uuid: "e119c267fd9a48718501f760222b923d" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 37421 } } peers { permanent_uuid: "d924c3a3ae57461c98d9e3c4470824cb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36039 } }
I20260811 01:49:29.182196 10891 raft_consensus.cc:374] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:29.182327 10891 raft_consensus.cc:729] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e119c267fd9a48718501f760222b923d, State: Initialized, Role: FOLLOWER
I20260811 01:49:29.183003 10891 consensus_queue.cc:260] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6391a80c14744499ac1146afac57af7d" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38783 } } peers { permanent_uuid: "e119c267fd9a48718501f760222b923d" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 37421 } } peers { permanent_uuid: "d924c3a3ae57461c98d9e3c4470824cb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36039 } }
I20260811 01:49:29.183840 10890 raft_consensus.cc:348] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6391a80c14744499ac1146afac57af7d" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38783 } } peers { permanent_uuid: "e119c267fd9a48718501f760222b923d" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 37421 } } peers { permanent_uuid: "d924c3a3ae57461c98d9e3c4470824cb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36039 } }
I20260811 01:49:29.184192 10712 heartbeater.cc:503] Master 127.21.214.126:34097 was elected leader, sending a full tablet report...
I20260811 01:49:29.184562 10890 raft_consensus.cc:374] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:29.184716 10890 raft_consensus.cc:729] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d924c3a3ae57461c98d9e3c4470824cb, State: Initialized, Role: FOLLOWER
I20260811 01:49:29.184935 10891 ts_tablet_manager.cc:1428] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d: Time spent starting tablet: real 0.011s user 0.008s sys 0.002s
I20260811 01:49:29.185376 10893 raft_consensus.cc:348] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6391a80c14744499ac1146afac57af7d" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38783 } } peers { permanent_uuid: "e119c267fd9a48718501f760222b923d" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 37421 } } peers { permanent_uuid: "d924c3a3ae57461c98d9e3c4470824cb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36039 } }
I20260811 01:49:29.185827 10893 raft_consensus.cc:374] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:29.185626 10890 consensus_queue.cc:260] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6391a80c14744499ac1146afac57af7d" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38783 } } peers { permanent_uuid: "e119c267fd9a48718501f760222b923d" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 37421 } } peers { permanent_uuid: "d924c3a3ae57461c98d9e3c4470824cb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36039 } }
I20260811 01:49:29.185938 10893 raft_consensus.cc:729] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6391a80c14744499ac1146afac57af7d, State: Initialized, Role: FOLLOWER
I20260811 01:49:29.186616 10893 consensus_queue.cc:260] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6391a80c14744499ac1146afac57af7d" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38783 } } peers { permanent_uuid: "e119c267fd9a48718501f760222b923d" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 37421 } } peers { permanent_uuid: "d924c3a3ae57461c98d9e3c4470824cb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36039 } }
I20260811 01:49:29.188040 10890 ts_tablet_manager.cc:1428] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb: Time spent starting tablet: real 0.015s user 0.008s sys 0.006s
I20260811 01:49:29.189028 10868 heartbeater.cc:503] Master 127.21.214.126:34097 was elected leader, sending a full tablet report...
I20260811 01:49:29.189780 10893 ts_tablet_manager.cc:1428] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d: Time spent starting tablet: real 0.015s user 0.012s sys 0.003s
W20260811 01:49:29.297461 10566 tablet.cc:2369] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:49:29.307957 10713 tablet.cc:2369] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:49:29.313637 10869 tablet.cc:2369] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:49:29.407703 10900 raft_consensus.cc:482] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:49:29.408025 10900 raft_consensus.cc:504] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6391a80c14744499ac1146afac57af7d" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38783 } } peers { permanent_uuid: "e119c267fd9a48718501f760222b923d" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 37421 } } peers { permanent_uuid: "d924c3a3ae57461c98d9e3c4470824cb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36039 } }
I20260811 01:49:29.409893 10900 leader_election.cc:288] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e119c267fd9a48718501f760222b923d (127.21.214.66:37421), d924c3a3ae57461c98d9e3c4470824cb (127.21.214.65:36039)
I20260811 01:49:29.416903 10887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:29.410528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:53901 (local address 127.21.214.66:37421)
0811 01:49:29.410791 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:29.410800 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:29.410820 (+ 20us) server_negotiation.cc:407] Connection header received
0811 01:49:29.410902 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:29.410907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:29.411000 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:29.411113 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:29.411801 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.412819 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.414836 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.415983 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.416109 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:29.416161 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:29.416572 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 01:49:29.416668 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 01:49:29.417165 10870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:29.410287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:37421 (local address 127.21.214.67:53901)
0811 01:49:29.410476 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:29.410494 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:29.410645 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:29.411343 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:29.411359 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:29.411386 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:29.411582 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:29.411595 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.412963 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.412969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:29.414648 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:29.414660 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.416153 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.416159 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:29.416391 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:29.416430 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 01:49:29.416542 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 01:49:29.416691 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 01:49:29.418051 10901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:29.410824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:36039 (local address 127.21.214.67:34129)
0811 01:49:29.411487 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:29.411506 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:29.411609 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:29.411962 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:29.411967 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:29.411992 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:29.412388 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:29.412400 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.413763 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.413767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:29.415501 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:29.415517 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.417401 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.417406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:29.417554 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:29.417577 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 01:49:29.417656 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 01:49:29.417757 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":328,"threads_started":1}
I20260811 01:49:29.418063 10888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:29.410926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:34129 (local address 127.21.214.65:36039)
0811 01:49:29.411209 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:29.411217 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:29.411604 (+ 387us) server_negotiation.cc:407] Connection header received
0811 01:49:29.411770 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:29.411776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:29.411848 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:29.411918 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:29.412565 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.413608 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.415762 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.416960 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.417132 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:29.417239 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:29.417738 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 01:49:29.417892 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 01:49:29.418478 10661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efe3f40a3b57484b8107f99c1be10d5d" candidate_uuid: "6391a80c14744499ac1146afac57af7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e119c267fd9a48718501f760222b923d" is_pre_election: true
I20260811 01:49:29.419054 10661 raft_consensus.cc:2454] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6391a80c14744499ac1146afac57af7d in term 0.
I20260811 01:49:29.419179 10510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efe3f40a3b57484b8107f99c1be10d5d" candidate_uuid: "6391a80c14744499ac1146afac57af7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d924c3a3ae57461c98d9e3c4470824cb" is_pre_election: true
I20260811 01:49:29.419782 10510 raft_consensus.cc:2454] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6391a80c14744499ac1146afac57af7d in term 0.
I20260811 01:49:29.419844 10739 leader_election.cc:302] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6391a80c14744499ac1146afac57af7d, e119c267fd9a48718501f760222b923d; no voters:
I20260811 01:49:29.420305 10900 raft_consensus.cc:2781] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:49:29.420493 10900 raft_consensus.cc:482] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:49:29.420619 10900 raft_consensus.cc:3037] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:29.423100 10900 raft_consensus.cc:504] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6391a80c14744499ac1146afac57af7d" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38783 } } peers { permanent_uuid: "e119c267fd9a48718501f760222b923d" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 37421 } } peers { permanent_uuid: "d924c3a3ae57461c98d9e3c4470824cb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36039 } }
I20260811 01:49:29.423754 10900 leader_election.cc:288] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [CANDIDATE]: Term 1 election: Requested vote from peers e119c267fd9a48718501f760222b923d (127.21.214.66:37421), d924c3a3ae57461c98d9e3c4470824cb (127.21.214.65:36039)
I20260811 01:49:29.424240 10661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efe3f40a3b57484b8107f99c1be10d5d" candidate_uuid: "6391a80c14744499ac1146afac57af7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e119c267fd9a48718501f760222b923d"
I20260811 01:49:29.424333 10510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efe3f40a3b57484b8107f99c1be10d5d" candidate_uuid: "6391a80c14744499ac1146afac57af7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d924c3a3ae57461c98d9e3c4470824cb"
I20260811 01:49:29.424465 10661 raft_consensus.cc:3037] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:29.424611 10510 raft_consensus.cc:3037] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:29.426894 10661 raft_consensus.cc:2454] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6391a80c14744499ac1146afac57af7d in term 1.
I20260811 01:49:29.426951 10510 raft_consensus.cc:2454] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6391a80c14744499ac1146afac57af7d in term 1.
I20260811 01:49:29.427395 10739 leader_election.cc:302] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6391a80c14744499ac1146afac57af7d, e119c267fd9a48718501f760222b923d; no voters:
I20260811 01:49:29.427716 10900 raft_consensus.cc:2781] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:49:29.428216 10900 raft_consensus.cc:686] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [term 1 LEADER]: Becoming Leader. State: Replica: 6391a80c14744499ac1146afac57af7d, State: Running, Role: LEADER
I20260811 01:49:29.428738 10900 consensus_queue.cc:237] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6391a80c14744499ac1146afac57af7d" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38783 } } peers { permanent_uuid: "e119c267fd9a48718501f760222b923d" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 37421 } } peers { permanent_uuid: "d924c3a3ae57461c98d9e3c4470824cb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36039 } }
I20260811 01:49:29.434211 10348 catalog_manager.cc:5495] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6391a80c14744499ac1146afac57af7d (127.21.214.67). New cstate: current_term: 1 leader_uuid: "6391a80c14744499ac1146afac57af7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6391a80c14744499ac1146afac57af7d" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 38783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e119c267fd9a48718501f760222b923d" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 37421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d924c3a3ae57461c98d9e3c4470824cb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36039 } health_report { overall_health: UNKNOWN } } }
I20260811 01:49:29.508929 10885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:29.502705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55180 (local address 127.21.214.67:38783)
0811 01:49:29.502914 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:29.502919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:29.502938 (+ 19us) server_negotiation.cc:407] Connection header received
0811 01:49:29.503149 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:29.503153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:29.503232 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:29.503367 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:49:29.503745 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.504592 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.506354 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.506769 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.506885 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:29.507439 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:49:29.508165 (+ 726us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:49:29.508307 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:29.508429 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 01:49:29.508505 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 01:49:29.518139 10661 raft_consensus.cc:1261] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d [term 1 FOLLOWER]: Refusing update from remote peer 6391a80c14744499ac1146afac57af7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:49:29.518139 10510 raft_consensus.cc:1261] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb [term 1 FOLLOWER]: Refusing update from remote peer 6391a80c14744499ac1146afac57af7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:49:29.519181 10900 consensus_queue.cc:1035] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e119c267fd9a48718501f760222b923d" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 37421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:49:29.519523 10904 consensus_queue.cc:1035] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d924c3a3ae57461c98d9e3c4470824cb" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:49:29.546713 10348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:49:29.547047 10348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:49:29.550958 10348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e06a86eeed9e45f68a324faac293521c: Sending DeleteTablet for 3 replicas of tablet efe3f40a3b57484b8107f99c1be10d5d
I20260811 01:49:29.552215 10787 tablet_service.cc:1514] Processing DeleteTablet for tablet efe3f40a3b57484b8107f99c1be10d5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:29 UTC) from {username='slave'} at 127.0.0.1:55172
I20260811 01:49:29.552559 10485 tablet_service.cc:1514] Processing DeleteTablet for tablet efe3f40a3b57484b8107f99c1be10d5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:29 UTC) from {username='slave'} at 127.0.0.1:41114
I20260811 01:49:29.553491 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 10419
I20260811 01:49:29.553304 10638 tablet_service.cc:1514] Processing DeleteTablet for tablet efe3f40a3b57484b8107f99c1be10d5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:29 UTC) from {username='slave'} at 127.0.0.1:38240
I20260811 01:49:29.565809 10920 tablet_replica.cc:330] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb: stopping tablet replica
I20260811 01:49:29.566576 10920 raft_consensus.cc:2229] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:49:29.568055 10919 tablet_replica.cc:330] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d: stopping tablet replica
I20260811 01:49:29.568645 10922 tablet_replica.cc:330] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d: stopping tablet replica
I20260811 01:49:29.567416 10920 raft_consensus.cc:2258] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:49:29.570997 10922 raft_consensus.cc:2229] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:49:29.571122 10920 ts_tablet_manager.cc:1905] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:49:29.571568 10919 raft_consensus.cc:2229] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:49:29.573848 10919 raft_consensus.cc:2258] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:49:29.575140 10922 raft_consensus.cc:2258] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:49:29.578388 10922 ts_tablet_manager.cc:1905] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:49:29.578943 10920 ts_tablet_manager.cc:1918] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:49:29.579185 10920 log.cc:1196] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/efe3f40a3b57484b8107f99c1be10d5d
I20260811 01:49:29.579907 10920 ts_tablet_manager.cc:1939] T efe3f40a3b57484b8107f99c1be10d5d P d924c3a3ae57461c98d9e3c4470824cb: Deleting consensus metadata
I20260811 01:49:29.581954 10888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:29.554895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41126 (local address 127.21.214.65:36039)
0811 01:49:29.555074 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:29.555080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:29.565523 (+ 10443us) server_negotiation.cc:407] Connection header received
0811 01:49:29.565892 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:29.565902 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:29.566081 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:29.566194 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:29.567109 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.568489 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.571827 (+ 3338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.572503 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.572605 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:29.579345 (+ 6740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:29.579359 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:29.579377 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:29.580179 (+ 802us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:29.580249 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:29.580275 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:29.580373 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:29.581455 (+ 1082us) server_negotiation.cc:299] Negotiation successful
0811 01:49:29.581599 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 01:49:29.582126 10337 catalog_manager.cc:4841] TS d924c3a3ae57461c98d9e3c4470824cb (127.21.214.65:36039): tablet efe3f40a3b57484b8107f99c1be10d5d (table test-table [id=92fef493cc104513ba343bcec3cc94b3]) successfully deleted
I20260811 01:49:29.583356 10560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:29.583356 10919 ts_tablet_manager.cc:1905] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:49:29.595952 10922 ts_tablet_manager.cc:1918] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:49:29.596110 10922 log.cc:1196] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/efe3f40a3b57484b8107f99c1be10d5d
I20260811 01:49:29.596657 10922 ts_tablet_manager.cc:1939] T efe3f40a3b57484b8107f99c1be10d5d P e119c267fd9a48718501f760222b923d: Deleting consensus metadata
I20260811 01:49:29.598645 10337 catalog_manager.cc:4841] TS e119c267fd9a48718501f760222b923d (127.21.214.66:37421): tablet efe3f40a3b57484b8107f99c1be10d5d (table test-table [id=92fef493cc104513ba343bcec3cc94b3]) successfully deleted
I20260811 01:49:29.601584 10919 ts_tablet_manager.cc:1918] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:49:29.601794 10919 log.cc:1196] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/efe3f40a3b57484b8107f99c1be10d5d
I20260811 01:49:29.602336 10919 ts_tablet_manager.cc:1939] T efe3f40a3b57484b8107f99c1be10d5d P 6391a80c14744499ac1146afac57af7d: Deleting consensus metadata
I20260811 01:49:29.604012 10337 catalog_manager.cc:4841] TS 6391a80c14744499ac1146afac57af7d (127.21.214.67:38783): tablet efe3f40a3b57484b8107f99c1be10d5d (table test-table [id=92fef493cc104513ba343bcec3cc94b3]) successfully deleted
I20260811 01:49:29.714428 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 10419
I20260811 01:49:29.737108 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 10569
I20260811 01:49:29.745537 10887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:29.738094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38244 (local address 127.21.214.66:37421)
0811 01:49:29.738311 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:29.738317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:29.738340 (+ 23us) server_negotiation.cc:407] Connection header received
0811 01:49:29.738477 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:29.738483 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:29.738599 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:29.738723 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:29.739109 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.740271 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.741818 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.742480 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.742631 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:29.743716 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:29.743729 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:29.743752 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:29.744615 (+ 863us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:29.744693 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:29.744721 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:29.744862 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:29.744970 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 01:49:29.745094 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 01:49:29.746373 10707 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:29.838831 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 10569
I20260811 01:49:29.861892 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 10716
I20260811 01:49:29.870396 10885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:29.862836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55186 (local address 127.21.214.67:38783)
0811 01:49:29.863037 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:29.863042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:29.863060 (+ 18us) server_negotiation.cc:407] Connection header received
0811 01:49:29.863264 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:29.863268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:29.863352 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:29.863440 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:29.863989 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.864807 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.866897 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:29.867343 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:29.867441 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:29.868842 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:29.868850 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:29.868866 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:29.869566 (+ 700us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:29.869621 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:29.869646 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:29.869736 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:29.869921 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 01:49:29.870010 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 01:49:29.871251 10863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:29.998027 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 10716
I20260811 01:49:30.023365 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 10314
I20260811 01:49:30.024654 10384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:30.136075 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 10314
2026-08-11T01:49:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3286 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:49:30 dist-test-slave-9gcw krb5kdc[10930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:49:30 dist-test-slave-9gcw krb5kdc[10930](info): set up 2 sockets
Aug 11 01:49:30 dist-test-slave-9gcw krb5kdc[10930](info): commencing operation
krb5kdc: starting...
W20260811 01:49:31.401423 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:49:31 dist-test-slave-9gcw krb5kdc[10930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:49:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:49:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:49:31.737471 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:42101
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:43191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:42101
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:49:32.106822 10946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:32.107141 10946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:32.107266 10946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:32.114705 10946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:49:32.114836 10946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:32.114928 10946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:32.115049 10946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:49:32.115111 10946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:49:32.125442 10946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:43191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:42101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:42101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:32.126932 10946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:32.128366 10946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:32.143110 10951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:32.143049 10952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:32.143783 10954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:32.144615 10946 server_base.cc:1034] running on GCE node
I20260811 01:49:32.145566 10946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:32.147159 10946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:32.148392 10946 hybrid_clock.cc:648] HybridClock initialized: now 1786412972148315 us; error 61 us; skew 500 ppm
I20260811 01:49:32.148773 10946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:32 dist-test-slave-9gcw krb5kdc[10930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412972, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:32.152300 10946 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:49:32.153964 10946 webserver.cc:460] Webserver started at http://127.21.214.126:39541/ using document root <none> and password file <none>
I20260811 01:49:32.154661 10946 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:32.154771 10946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:32.155064 10946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:32.158322 10946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "3712cb77bae64e91b6a818cb3af457a3"
format_stamp: "Formatted at 2026-08-11 01:49:32 on dist-test-slave-9gcw"
server_key: "ecbba1541b6122f18b444ec7e644ef08"
server_key_iv: "e0f144dcf7e21426fc398ba72d141784"
server_key_version: "encryptionkey@0"
I20260811 01:49:32.159076 10946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "3712cb77bae64e91b6a818cb3af457a3"
format_stamp: "Formatted at 2026-08-11 01:49:32 on dist-test-slave-9gcw"
server_key: "ecbba1541b6122f18b444ec7e644ef08"
server_key_iv: "e0f144dcf7e21426fc398ba72d141784"
server_key_version: "encryptionkey@0"
I20260811 01:49:32.164754 10946 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 01:49:32.168401 10963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:32.169863 10946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 01:49:32.170068 10946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "3712cb77bae64e91b6a818cb3af457a3"
format_stamp: "Formatted at 2026-08-11 01:49:32 on dist-test-slave-9gcw"
server_key: "ecbba1541b6122f18b444ec7e644ef08"
server_key_iv: "e0f144dcf7e21426fc398ba72d141784"
server_key_version: "encryptionkey@0"
I20260811 01:49:32.170346 10946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:32.201314 10946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:32.205727 10946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:32.206058 10946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:32.225572 10946 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:42101
I20260811 01:49:32.225584 11022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:42101 every 8 connection(s)
I20260811 01:49:32.227022 10946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:49:32.230903 11023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:32.232632 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 10946
I20260811 01:49:32.232889 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:49:32.233507 22361 external_mini_cluster.cc:1428] Setting key c6918b7e314b08dba16e64edcc6ec522
I20260811 01:49:32.243654 11023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3: Bootstrap starting.
I20260811 01:49:32.247756 11023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:32.249133 11023 log.cc:824] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3: Log is configured to *not* fsync() on all Append() calls
Aug 11 01:49:32 dist-test-slave-9gcw krb5kdc[10930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:49:32.252522 11023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3: No bootstrap required, opened a new log
I20260811 01:49:32.258888 11026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:32.235494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34982 (local address 127.21.214.126:42101)
0811 01:49:32.236403 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:32.236415 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:32.236462 (+ 47us) server_negotiation.cc:407] Connection header received
0811 01:49:32.238241 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:32.238267 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:32.238657 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:32.238988 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:32.240985 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:32.242473 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:32.244490 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:32.245306 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:32.245449 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:32.251027 (+ 5578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:32.251035 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:32.251051 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:32.251092 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:32.254394 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:32.255154 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:32.255173 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:32.255180 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:32.255380 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:32.255804 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:32.255811 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:32.255822 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:32.256382 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:32.256722 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:32.257270 (+ 548us) server_negotiation.cc:299] Negotiation successful
0811 01:49:32.257762 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":416,"threads_started":1}
I20260811 01:49:32.261261 11023 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3712cb77bae64e91b6a818cb3af457a3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42101 } }
I20260811 01:49:32.261754 11023 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:32.261891 11023 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3712cb77bae64e91b6a818cb3af457a3, State: Initialized, Role: FOLLOWER
I20260811 01:49:32.262574 11023 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3712cb77bae64e91b6a818cb3af457a3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42101 } }
I20260811 01:49:32.262849 11023 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:49:32.262956 11023 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:49:32.263089 11023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:32.265357 11023 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3712cb77bae64e91b6a818cb3af457a3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42101 } }
I20260811 01:49:32.265892 11023 leader_election.cc:302] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3712cb77bae64e91b6a818cb3af457a3; no voters:
I20260811 01:49:32.266480 11023 leader_election.cc:288] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:49:32.266662 11028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:49:32.268115 11028 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3 [term 1 LEADER]: Becoming Leader. State: Replica: 3712cb77bae64e91b6a818cb3af457a3, State: Running, Role: LEADER
I20260811 01:49:32.268720 11028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3712cb77bae64e91b6a818cb3af457a3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42101 } }
I20260811 01:49:32.268987 11023 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:49:32.271936 11030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3712cb77bae64e91b6a818cb3af457a3. Latest consensus state: current_term: 1 leader_uuid: "3712cb77bae64e91b6a818cb3af457a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3712cb77bae64e91b6a818cb3af457a3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42101 } } }
I20260811 01:49:32.271960 11029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3712cb77bae64e91b6a818cb3af457a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3712cb77bae64e91b6a818cb3af457a3" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 42101 } } }
I20260811 01:49:32.272493 11029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3 [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:32.273185 11030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3 [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:32.273459 11039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:49:32.281339 11039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:49:32.292671 11039 catalog_manager.cc:1347] Generated new cluster ID: 42926637059c4c2597c69b7a6dcb5622
I20260811 01:49:32.292837 11039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:49:32.317651 11039 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:49:32.319208 11039 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:49:32.329068 11039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3: Generated new TSK 0
I20260811 01:49:32.330060 11039 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
I20260811 01:49:32.405787 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42101
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:43191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:49:32.770823 11055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:32.771124 11055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:32.771279 11055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:32.779089 11055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:32.779223 11055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:32.779434 11055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:49:32.790107 11055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:43191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:32.791524 11055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:32.792924 11055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:32.806586 11060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:32.806604 11061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:32.806540 11063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:32.807511 11055 server_base.cc:1034] running on GCE node
I20260811 01:49:32.808156 11055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:32.809213 11055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:32.810485 11055 hybrid_clock.cc:648] HybridClock initialized: now 1786412972810432 us; error 36 us; skew 500 ppm
I20260811 01:49:32.810909 11055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:32 dist-test-slave-9gcw krb5kdc[10930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412972, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:32.820263 11055 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:49:32.822085 11055 webserver.cc:460] Webserver started at http://127.21.214.65:42467/ using document root <none> and password file <none>
I20260811 01:49:32.822926 11055 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:32.823098 11055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:32.823482 11055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:32.826969 11055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "f5f86857c9d64a299e00bc5fa8953150"
format_stamp: "Formatted at 2026-08-11 01:49:32 on dist-test-slave-9gcw"
server_key: "bc9a2d34448552899e4e468fcaea0333"
server_key_iv: "78724a7d6a74f343d51d42f0577ebdb2"
server_key_version: "encryptionkey@0"
I20260811 01:49:32.827880 11055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "f5f86857c9d64a299e00bc5fa8953150"
format_stamp: "Formatted at 2026-08-11 01:49:32 on dist-test-slave-9gcw"
server_key: "bc9a2d34448552899e4e468fcaea0333"
server_key_iv: "78724a7d6a74f343d51d42f0577ebdb2"
server_key_version: "encryptionkey@0"
I20260811 01:49:32.834219 11055 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 01:49:32.838179 11071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:32.839612 11055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260811 01:49:32.839821 11055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "f5f86857c9d64a299e00bc5fa8953150"
format_stamp: "Formatted at 2026-08-11 01:49:32 on dist-test-slave-9gcw"
server_key: "bc9a2d34448552899e4e468fcaea0333"
server_key_iv: "78724a7d6a74f343d51d42f0577ebdb2"
server_key_version: "encryptionkey@0"
I20260811 01:49:32.840047 11055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:32.854491 11055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:32.858763 11055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:32.859084 11055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:32.860122 11055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:32.861586 11055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:32.861696 11055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:32.861853 11055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:32.861937 11055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:32.898100 11055 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:36539
I20260811 01:49:32.898138 11203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:36539 every 8 connection(s)
I20260811 01:49:32.899562 11055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:49:32.908913 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 11055
I20260811 01:49:32.909118 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:49:32.909731 22361 external_mini_cluster.cc:1428] Setting key 96b0071e6eaf78a3b4646ca5e0c02919
Aug 11 01:49:32 dist-test-slave-9gcw krb5kdc[10930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412972, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:49:32.922106 11207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:32.902547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:54417 (local address 127.21.214.126:42101)
0811 01:49:32.903066 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:32.903073 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:32.904117 (+ 1044us) server_negotiation.cc:407] Connection header received
0811 01:49:32.906264 (+ 2147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:32.906268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:32.906338 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:32.906444 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:32.908246 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:32.909025 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:32.910892 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:32.911349 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:32.911496 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:32.915818 (+ 4322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:32.915823 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:32.915827 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:32.915863 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:32.918573 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:32.919605 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:32.919609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:32.919612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:32.919731 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:32.920270 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:32.920273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:32.920275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:32.920529 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:32.920651 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:32.921661 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0811 01:49:32.921843 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":379,"thread_start_us":255,"threads_started":1}
I20260811 01:49:32.923797 11208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:32.903113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:42101 (local address 127.21.214.65:54417)
0811 01:49:32.903926 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:32.903982 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:32.905961 (+ 1979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:32.906762 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:32.906772 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:32.907413 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:49:32.908051 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:32.908071 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:32.909210 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:32.909218 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:32.910728 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:32.910748 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:32.911596 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:32.911605 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:32.911867 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:32.912587 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:49:32.912610 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:49:32.915560 (+ 2950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:49:32.918741 (+ 3181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:49:32.918754 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:49:32.918776 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:49:32.919472 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:49:32.919841 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:49:32.919848 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:49:32.919854 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:49:32.920144 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:49:32.920664 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:49:32.920675 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:49:32.921218 (+ 543us) client_negotiation.cc:769] Sending connection context
0811 01:49:32.921548 (+ 330us) client_negotiation.cc:240] Negotiation successful
0811 01:49:32.922184 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":291,"threads_started":1}
I20260811 01:49:32.926005 11205 heartbeater.cc:347] Connected to a master server at 127.21.214.126:42101
I20260811 01:49:32.926424 11205 heartbeater.cc:464] Registering TS with master...
I20260811 01:49:32.927470 11205 heartbeater.cc:511] Master 127.21.214.126:42101 requested a full tablet report, sending...
I20260811 01:49:32.930194 10983 ts_manager.cc:194] Registered new tserver with Master: f5f86857c9d64a299e00bc5fa8953150 (127.21.214.65:36539)
I20260811 01:49:32.932462 10983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:54417
WARNING: no policy specified for kudu/127.21.214.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
I20260811 01:49:32.984898 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42101
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:43191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:49:33.373574 11211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:33.373944 11211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:33.374114 11211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:33.381614 11211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:33.381795 11211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:33.381969 11211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:49:33.392161 11211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:43191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:33.393601 11211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:33.395020 11211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:33.408771 11220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:33.408810 11217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:33.408856 11216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:33.410418 11211 server_base.cc:1034] running on GCE node
I20260811 01:49:33.411134 11211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:33.412204 11211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:33.413437 11211 hybrid_clock.cc:648] HybridClock initialized: now 1786412973413385 us; error 46 us; skew 500 ppm
I20260811 01:49:33.413789 11211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:33 dist-test-slave-9gcw krb5kdc[10930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412973, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:33.417601 11211 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:49:33.419210 11211 webserver.cc:460] Webserver started at http://127.21.214.66:46811/ using document root <none> and password file <none>
I20260811 01:49:33.420013 11211 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:33.420167 11211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:33.420506 11211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:33.423763 11211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "32bd305033d540d69271ba0c770bab77"
format_stamp: "Formatted at 2026-08-11 01:49:33 on dist-test-slave-9gcw"
server_key: "1d7e1c04b42040f9609772a064e48564"
server_key_iv: "ea2c243b32d7d61b06264d92fd08a09a"
server_key_version: "encryptionkey@0"
I20260811 01:49:33.424575 11211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "32bd305033d540d69271ba0c770bab77"
format_stamp: "Formatted at 2026-08-11 01:49:33 on dist-test-slave-9gcw"
server_key: "1d7e1c04b42040f9609772a064e48564"
server_key_iv: "ea2c243b32d7d61b06264d92fd08a09a"
server_key_version: "encryptionkey@0"
I20260811 01:49:33.430130 11211 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 01:49:33.433951 11227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:33.435344 11211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:49:33.435527 11211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "32bd305033d540d69271ba0c770bab77"
format_stamp: "Formatted at 2026-08-11 01:49:33 on dist-test-slave-9gcw"
server_key: "1d7e1c04b42040f9609772a064e48564"
server_key_iv: "ea2c243b32d7d61b06264d92fd08a09a"
server_key_version: "encryptionkey@0"
I20260811 01:49:33.435734 11211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:33.448707 11211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:33.452633 11211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:33.452950 11211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:33.453984 11211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:33.455348 11211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:33.455454 11211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:33.455606 11211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:33.455682 11211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:33.490996 11211 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:42095
I20260811 01:49:33.491041 11369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:42095 every 8 connection(s)
I20260811 01:49:33.492357 11211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:49:33.500048 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 11211
I20260811 01:49:33.500267 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:49:33.500767 22361 external_mini_cluster.cc:1428] Setting key 3754362e9e0a6ad34abd588a4eceaf4e
Aug 11 01:49:33 dist-test-slave-9gcw krb5kdc[10930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412973, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:49:33.514967 11373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:33.495056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:35745 (local address 127.21.214.126:42101)
0811 01:49:33.495530 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:33.495535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:33.496552 (+ 1017us) server_negotiation.cc:407] Connection header received
0811 01:49:33.498460 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:33.498464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:33.498532 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:33.498640 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:33.500491 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:33.501377 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:33.502979 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:33.503392 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:33.503486 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:33.507816 (+ 4330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:33.507824 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:33.507832 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:33.507879 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:33.511563 (+ 3684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:33.512299 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:33.512308 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:33.512314 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:33.512488 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:33.512904 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:33.512911 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:33.512917 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:33.513337 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:33.513497 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:33.514215 (+ 718us) server_negotiation.cc:299] Negotiation successful
0811 01:49:33.514474 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":237,"threads_started":1}
I20260811 01:49:33.515669 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:33.495676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:42101 (local address 127.21.214.66:35745)
0811 01:49:33.496381 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:33.496421 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:33.498191 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:33.498877 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:33.498885 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:33.499398 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:49:33.500230 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:33.500262 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:33.501540 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:33.501544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:33.502830 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:33.502845 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:33.503629 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:33.503640 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:33.504003 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:33.504775 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:49:33.504811 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:49:33.507565 (+ 2754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:49:33.511722 (+ 4157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:49:33.511729 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:49:33.511742 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:49:33.512172 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:49:33.512600 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:49:33.512603 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:49:33.512605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:49:33.512789 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:49:33.513469 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:49:33.513475 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:49:33.513823 (+ 348us) client_negotiation.cc:769] Sending connection context
0811 01:49:33.514060 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 01:49:33.514476 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":258,"threads_started":1}
I20260811 01:49:33.517385 11370 heartbeater.cc:347] Connected to a master server at 127.21.214.126:42101
I20260811 01:49:33.517771 11370 heartbeater.cc:464] Registering TS with master...
I20260811 01:49:33.518666 11370 heartbeater.cc:511] Master 127.21.214.126:42101 requested a full tablet report, sending...
I20260811 01:49:33.520519 10983 ts_manager.cc:194] Registered new tserver with Master: 32bd305033d540d69271ba0c770bab77 (127.21.214.66:42095)
I20260811 01:49:33.521826 10983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:35745
WARNING: no policy specified for kudu/127.21.214.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
I20260811 01:49:33.572774 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42101
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:43191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:49:33.925467 11377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:33.925832 11377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:33.925992 11377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:33.933568 11377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:33.933723 11377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:33.933905 11377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:49:33.935969 11205 heartbeater.cc:503] Master 127.21.214.126:42101 was elected leader, sending a full tablet report...
I20260811 01:49:33.944830 11377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:43191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:42101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:33.946214 11377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:33.947536 11377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:33.961408 11384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:33.961841 11386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:33.962126 11383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:33.963524 11377 server_base.cc:1034] running on GCE node
I20260811 01:49:33.964215 11377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:33.965418 11377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:33.966634 11377 hybrid_clock.cc:648] HybridClock initialized: now 1786412973966561 us; error 61 us; skew 500 ppm
I20260811 01:49:33.966995 11377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:33 dist-test-slave-9gcw krb5kdc[10930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412973, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:33.970460 11377 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:49:33.972026 11377 webserver.cc:460] Webserver started at http://127.21.214.67:34807/ using document root <none> and password file <none>
I20260811 01:49:33.972798 11377 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:33.972952 11377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:33.973362 11377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:33.976594 11377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "a0280de2a4c740c99ded9d1895f21d4f"
format_stamp: "Formatted at 2026-08-11 01:49:33 on dist-test-slave-9gcw"
server_key: "67c7eccd47d486bd0162206c2d7c99a2"
server_key_iv: "d65c4b0c05d2b7839482cb886498787c"
server_key_version: "encryptionkey@0"
I20260811 01:49:33.977487 11377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "a0280de2a4c740c99ded9d1895f21d4f"
format_stamp: "Formatted at 2026-08-11 01:49:33 on dist-test-slave-9gcw"
server_key: "67c7eccd47d486bd0162206c2d7c99a2"
server_key_iv: "d65c4b0c05d2b7839482cb886498787c"
server_key_version: "encryptionkey@0"
I20260811 01:49:33.982990 11377 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 01:49:33.986867 11393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:33.988371 11377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 01:49:33.988548 11377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "a0280de2a4c740c99ded9d1895f21d4f"
format_stamp: "Formatted at 2026-08-11 01:49:33 on dist-test-slave-9gcw"
server_key: "67c7eccd47d486bd0162206c2d7c99a2"
server_key_iv: "d65c4b0c05d2b7839482cb886498787c"
server_key_version: "encryptionkey@0"
I20260811 01:49:33.988770 11377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:34.027834 11377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:34.032387 11377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:34.032734 11377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:34.033838 11377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:34.035348 11377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:34.035456 11377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:34.035611 11377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:34.035684 11377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:34.073866 11377 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:45487
I20260811 01:49:34.073972 11526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:45487 every 8 connection(s)
I20260811 01:49:34.075462 11377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:49:34.075883 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 11377
I20260811 01:49:34.076095 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:49:34.076756 22361 external_mini_cluster.cc:1428] Setting key 4dedc6e76dfeac972b480a460756b388
Aug 11 01:49:34 dist-test-slave-9gcw krb5kdc[10930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412973, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:49:34.097646 11529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:34.078399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:57329 (local address 127.21.214.126:42101)
0811 01:49:34.078904 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:34.078910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:34.081242 (+ 2332us) server_negotiation.cc:407] Connection header received
0811 01:49:34.082323 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:34.082328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:34.082421 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:34.082553 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:34.084379 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.085253 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.087055 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.087490 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.087633 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:34.091435 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:34.091439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:34.091443 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:34.091478 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:34.094668 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:34.095432 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:34.095437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:34.095439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:34.095575 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:34.095993 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:34.095997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:34.095999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:34.096300 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:34.096423 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:34.097089 (+ 666us) server_negotiation.cc:299] Negotiation successful
0811 01:49:34.097361 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":233,"threads_started":1}
I20260811 01:49:34.098681 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:34.079197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:42101 (local address 127.21.214.67:57329)
0811 01:49:34.080170 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:34.080234 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:34.082053 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:34.082820 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:34.082829 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:34.083587 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:49:34.084195 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:34.084218 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.085420 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.085425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:34.086898 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:34.086911 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.087744 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.087753 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:34.087998 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:34.088666 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:49:34.088689 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:49:34.091221 (+ 2532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:49:34.094818 (+ 3597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:49:34.094826 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:49:34.094841 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:49:34.095298 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:49:34.095678 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:49:34.095682 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:49:34.095684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:49:34.095885 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:49:34.096411 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:49:34.096417 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:49:34.096757 (+ 340us) client_negotiation.cc:769] Sending connection context
0811 01:49:34.097000 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 01:49:34.097489 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":315,"threads_started":1}
I20260811 01:49:34.100083 11527 heartbeater.cc:347] Connected to a master server at 127.21.214.126:42101
I20260811 01:49:34.100374 11527 heartbeater.cc:464] Registering TS with master...
I20260811 01:49:34.101099 11527 heartbeater.cc:511] Master 127.21.214.126:42101 requested a full tablet report, sending...
I20260811 01:49:34.102591 10983 ts_manager.cc:194] Registered new tserver with Master: a0280de2a4c740c99ded9d1895f21d4f (127.21.214.67:45487)
I20260811 01:49:34.103510 10983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:57329
I20260811 01:49:34.105804 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:49:34.123212 11529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:34.110052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55842 (local address 127.21.214.126:42101)
0811 01:49:34.110291 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:34.110298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:34.110557 (+ 259us) server_negotiation.cc:407] Connection header received
0811 01:49:34.110755 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:34.110759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:34.110823 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:34.110909 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:34.112292 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.113083 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.114510 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.114957 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.115060 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:34.117865 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:34.117870 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:34.117873 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:34.117908 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:34.120495 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:34.121336 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:34.121340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:34.121343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:34.121470 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:34.121867 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:34.121870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:34.121872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:34.122164 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:34.122366 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:34.122688 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 01:49:34.122846 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 01:49:34.127410 10983 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:49:34.134441 10983 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:49:34.173014 11540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:34.147597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:45487 (local address 127.0.0.1:57856)
0811 01:49:34.148580 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:34.148627 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:34.148838 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:34.162216 (+ 13378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:34.162223 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:34.162254 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:34.162885 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:34.162895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.166649 (+ 3754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.166657 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:34.169613 (+ 2956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:34.169628 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.171667 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.171678 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:34.171971 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:34.172061 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 01:49:34.172199 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 01:49:34.172444 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"mutex_wait_us":266,"thread_start_us":550,"threads_started":1}
I20260811 01:49:34.173717 11545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:34.149196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:36539 (local address 127.0.0.1:57962)
0811 01:49:34.162596 (+ 13400us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:34.162618 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:34.162766 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:34.163583 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:34.163587 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:34.163611 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:34.163998 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:34.164012 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.167805 (+ 3793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.167811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:34.170650 (+ 2839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:34.170690 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.172820 (+ 2130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.172826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:34.173018 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:34.173057 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 01:49:34.173215 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 01:49:34.173343 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13260,"mutex_wait_us":2,"thread_start_us":13057,"threads_started":1}
I20260811 01:49:34.174218 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:34.147881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57856 (local address 127.21.214.67:45487)
0811 01:49:34.161583 (+ 13702us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:34.161594 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:34.161630 (+ 36us) server_negotiation.cc:407] Connection header received
0811 01:49:34.161736 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:34.161743 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:34.161984 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:34.162696 (+ 712us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:34.163384 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.166468 (+ 3084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.169810 (+ 3342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.171426 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.173301 (+ 1875us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:34.173554 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:34.173628 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 01:49:34.173810 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13509,"thread_start_us":13338,"threads_started":1}
I20260811 01:49:34.174516 11546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:34.149892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57962 (local address 127.21.214.65:36539)
0811 01:49:34.162011 (+ 12119us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:34.162023 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:34.162826 (+ 803us) server_negotiation.cc:407] Connection header received
0811 01:49:34.162937 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:34.162942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:34.163095 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:34.163306 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:34.164260 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.167597 (+ 3337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.170902 (+ 3305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.172586 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.172745 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:34.173013 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:34.173486 (+ 473us) server_negotiation.cc:299] Negotiation successful
0811 01:49:34.173688 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11912,"thread_start_us":11757,"threads_started":1}
I20260811 01:49:34.176349 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:34.148304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:42095 (local address 127.0.0.1:36062)
0811 01:49:34.160658 (+ 12354us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:34.160679 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:34.160879 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:34.162005 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:34.162017 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:34.162051 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:34.162723 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:34.162739 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.165100 (+ 2361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.165106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:34.169902 (+ 4796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:34.169915 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.175495 (+ 5580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.175501 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:34.175679 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:34.175716 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 01:49:34.175810 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 01:49:34.175948 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12243,"mutex_wait_us":69,"thread_start_us":12562,"threads_started":1}
I20260811 01:49:34.177107 11544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:34.148833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36062 (local address 127.21.214.66:42095)
0811 01:49:34.159716 (+ 10883us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:34.159724 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:34.160821 (+ 1097us) server_negotiation.cc:407] Connection header received
0811 01:49:34.161112 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:34.161125 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:34.161484 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:34.161706 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:34.163306 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.164934 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.173029 (+ 8095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.175006 (+ 1977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.175145 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:34.175354 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:34.176578 (+ 1224us) server_negotiation.cc:299] Negotiation successful
0811 01:49:34.176758 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10701,"thread_start_us":10548,"threads_started":1}
I20260811 01:49:34.178891 11447 tablet_service.cc:1467] Processing CreateTablet for tablet c08a74710f0a4852b1b4a621a2a6107c (DEFAULT_TABLE table=test-table [id=06223a3b5e664dcf98dbe1b391782523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:49:34.179037 11125 tablet_service.cc:1467] Processing CreateTablet for tablet c08a74710f0a4852b1b4a621a2a6107c (DEFAULT_TABLE table=test-table [id=06223a3b5e664dcf98dbe1b391782523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:49:34.180267 11447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c08a74710f0a4852b1b4a621a2a6107c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:34.180588 11125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c08a74710f0a4852b1b4a621a2a6107c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:34.180629 11288 tablet_service.cc:1467] Processing CreateTablet for tablet c08a74710f0a4852b1b4a621a2a6107c (DEFAULT_TABLE table=test-table [id=06223a3b5e664dcf98dbe1b391782523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:49:34.181948 11288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c08a74710f0a4852b1b4a621a2a6107c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:34.192595 11547 tablet_bootstrap.cc:492] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f: Bootstrap starting.
I20260811 01:49:34.195469 11548 tablet_bootstrap.cc:492] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150: Bootstrap starting.
I20260811 01:49:34.197847 11547 tablet_bootstrap.cc:654] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:34.197919 11549 tablet_bootstrap.cc:492] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77: Bootstrap starting.
I20260811 01:49:34.199175 11548 tablet_bootstrap.cc:654] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:34.199410 11547 log.cc:824] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:34.200508 11548 log.cc:824] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:34.201305 11549 tablet_bootstrap.cc:654] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:34.202252 11549 log.cc:824] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:34.203444 11547 tablet_bootstrap.cc:492] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f: No bootstrap required, opened a new log
I20260811 01:49:34.203802 11547 ts_tablet_manager.cc:1397] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260811 01:49:34.205418 11549 tablet_bootstrap.cc:492] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77: No bootstrap required, opened a new log
I20260811 01:49:34.205677 11549 ts_tablet_manager.cc:1397] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 01:49:34.212114 11548 tablet_bootstrap.cc:492] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150: No bootstrap required, opened a new log
I20260811 01:49:34.212425 11548 ts_tablet_manager.cc:1397] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.014s
I20260811 01:49:34.214037 11549 raft_consensus.cc:348] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0280de2a4c740c99ded9d1895f21d4f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45487 } } peers { permanent_uuid: "f5f86857c9d64a299e00bc5fa8953150" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36539 } } peers { permanent_uuid: "32bd305033d540d69271ba0c770bab77" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42095 } }
I20260811 01:49:34.214444 11549 raft_consensus.cc:374] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:34.214573 11549 raft_consensus.cc:729] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32bd305033d540d69271ba0c770bab77, State: Initialized, Role: FOLLOWER
I20260811 01:49:34.215233 11549 consensus_queue.cc:260] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0280de2a4c740c99ded9d1895f21d4f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45487 } } peers { permanent_uuid: "f5f86857c9d64a299e00bc5fa8953150" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36539 } } peers { permanent_uuid: "32bd305033d540d69271ba0c770bab77" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42095 } }
I20260811 01:49:34.215893 11547 raft_consensus.cc:348] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0280de2a4c740c99ded9d1895f21d4f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45487 } } peers { permanent_uuid: "f5f86857c9d64a299e00bc5fa8953150" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36539 } } peers { permanent_uuid: "32bd305033d540d69271ba0c770bab77" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42095 } }
I20260811 01:49:34.216322 11547 raft_consensus.cc:374] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:34.216408 11547 raft_consensus.cc:729] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0280de2a4c740c99ded9d1895f21d4f, State: Initialized, Role: FOLLOWER
I20260811 01:49:34.216989 11547 consensus_queue.cc:260] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0280de2a4c740c99ded9d1895f21d4f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45487 } } peers { permanent_uuid: "f5f86857c9d64a299e00bc5fa8953150" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36539 } } peers { permanent_uuid: "32bd305033d540d69271ba0c770bab77" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42095 } }
I20260811 01:49:34.218779 11370 heartbeater.cc:503] Master 127.21.214.126:42101 was elected leader, sending a full tablet report...
I20260811 01:49:34.219331 11549 ts_tablet_manager.cc:1428] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260811 01:49:34.220731 11527 heartbeater.cc:503] Master 127.21.214.126:42101 was elected leader, sending a full tablet report...
I20260811 01:49:34.221397 11547 ts_tablet_manager.cc:1428] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f: Time spent starting tablet: real 0.017s user 0.015s sys 0.004s
I20260811 01:49:34.223140 11548 raft_consensus.cc:348] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0280de2a4c740c99ded9d1895f21d4f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45487 } } peers { permanent_uuid: "f5f86857c9d64a299e00bc5fa8953150" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36539 } } peers { permanent_uuid: "32bd305033d540d69271ba0c770bab77" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42095 } }
I20260811 01:49:34.223569 11548 raft_consensus.cc:374] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:34.223682 11548 raft_consensus.cc:729] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f86857c9d64a299e00bc5fa8953150, State: Initialized, Role: FOLLOWER
I20260811 01:49:34.224290 11548 consensus_queue.cc:260] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0280de2a4c740c99ded9d1895f21d4f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45487 } } peers { permanent_uuid: "f5f86857c9d64a299e00bc5fa8953150" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36539 } } peers { permanent_uuid: "32bd305033d540d69271ba0c770bab77" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42095 } }
I20260811 01:49:34.230716 11548 ts_tablet_manager.cc:1428] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150: Time spent starting tablet: real 0.018s user 0.019s sys 0.001s
W20260811 01:49:34.244439 11371 tablet.cc:2369] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:49:34.309194 11556 raft_consensus.cc:482] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:49:34.309562 11556 raft_consensus.cc:504] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0280de2a4c740c99ded9d1895f21d4f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45487 } } peers { permanent_uuid: "f5f86857c9d64a299e00bc5fa8953150" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36539 } } peers { permanent_uuid: "32bd305033d540d69271ba0c770bab77" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42095 } }
I20260811 01:49:34.311419 11556 leader_election.cc:288] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5f86857c9d64a299e00bc5fa8953150 (127.21.214.65:36539), 32bd305033d540d69271ba0c770bab77 (127.21.214.66:42095)
I20260811 01:49:34.317368 11546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:34.311993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:60257 (local address 127.21.214.65:36539)
0811 01:49:34.312234 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:34.312240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:34.312265 (+ 25us) server_negotiation.cc:407] Connection header received
0811 01:49:34.312490 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:34.312494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:34.312555 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:34.312693 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:34.313208 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.313944 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.315592 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.316391 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.316574 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:34.316671 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:34.316927 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 01:49:34.317056 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 01:49:34.317355 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:34.311976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:36539 (local address 127.21.214.67:60257)
0811 01:49:34.312173 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:34.312194 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:34.312336 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:34.312725 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:34.312730 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:34.312772 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:34.312990 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:34.312998 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.314108 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.314119 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:34.315423 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:34.315438 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.316556 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.316563 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:34.316760 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:34.316785 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 01:49:34.316889 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 01:49:34.317031 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 01:49:34.318552 11544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:34.312017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:57023 (local address 127.21.214.66:42095)
0811 01:49:34.312232 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:34.312239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:34.313175 (+ 936us) server_negotiation.cc:407] Connection header received
0811 01:49:34.313386 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:34.313395 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:34.313458 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:34.313560 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:34.314126 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.314841 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.316546 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.317441 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.317618 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:34.317692 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:34.318158 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 01:49:34.318291 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 01:49:34.318645 11558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:34.311987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:42095 (local address 127.21.214.67:57023)
0811 01:49:34.313036 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:34.313050 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:34.313229 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:34.313580 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:34.313585 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:34.313609 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:34.313952 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:34.313961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.315007 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.315012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:34.316370 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:34.316381 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.317874 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.317879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:34.317996 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:34.318017 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 01:49:34.318126 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 01:49:34.318251 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":930,"mutex_wait_us":233,"thread_start_us":479,"threads_started":1}
I20260811 01:49:34.318553 11147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c08a74710f0a4852b1b4a621a2a6107c" candidate_uuid: "a0280de2a4c740c99ded9d1895f21d4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f86857c9d64a299e00bc5fa8953150" is_pre_election: true
I20260811 01:49:34.319322 11147 raft_consensus.cc:2454] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0280de2a4c740c99ded9d1895f21d4f in term 0.
I20260811 01:49:34.319823 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c08a74710f0a4852b1b4a621a2a6107c" candidate_uuid: "a0280de2a4c740c99ded9d1895f21d4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32bd305033d540d69271ba0c770bab77" is_pre_election: true
I20260811 01:49:34.320150 11394 leader_election.cc:302] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0280de2a4c740c99ded9d1895f21d4f, f5f86857c9d64a299e00bc5fa8953150; no voters:
I20260811 01:49:34.320333 11313 raft_consensus.cc:2454] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0280de2a4c740c99ded9d1895f21d4f in term 0.
I20260811 01:49:34.320592 11556 raft_consensus.cc:2781] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:49:34.320739 11556 raft_consensus.cc:482] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:49:34.320850 11556 raft_consensus.cc:3037] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:34.323374 11556 raft_consensus.cc:504] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0280de2a4c740c99ded9d1895f21d4f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45487 } } peers { permanent_uuid: "f5f86857c9d64a299e00bc5fa8953150" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36539 } } peers { permanent_uuid: "32bd305033d540d69271ba0c770bab77" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42095 } }
I20260811 01:49:34.324011 11556 leader_election.cc:288] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [CANDIDATE]: Term 1 election: Requested vote from peers f5f86857c9d64a299e00bc5fa8953150 (127.21.214.65:36539), 32bd305033d540d69271ba0c770bab77 (127.21.214.66:42095)
I20260811 01:49:34.324497 11147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c08a74710f0a4852b1b4a621a2a6107c" candidate_uuid: "a0280de2a4c740c99ded9d1895f21d4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f86857c9d64a299e00bc5fa8953150"
I20260811 01:49:34.324595 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c08a74710f0a4852b1b4a621a2a6107c" candidate_uuid: "a0280de2a4c740c99ded9d1895f21d4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32bd305033d540d69271ba0c770bab77"
I20260811 01:49:34.324738 11147 raft_consensus.cc:3037] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:34.324803 11313 raft_consensus.cc:3037] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77 [term 0 FOLLOWER]: Advancing to term 1
W20260811 01:49:34.327306 11528 tablet.cc:2369] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:49:34.328131 11147 raft_consensus.cc:2454] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0280de2a4c740c99ded9d1895f21d4f in term 1.
I20260811 01:49:34.328131 11313 raft_consensus.cc:2454] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0280de2a4c740c99ded9d1895f21d4f in term 1.
I20260811 01:49:34.328568 11394 leader_election.cc:302] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0280de2a4c740c99ded9d1895f21d4f, f5f86857c9d64a299e00bc5fa8953150; no voters:
I20260811 01:49:34.328902 11556 raft_consensus.cc:2781] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:49:34.329380 11556 raft_consensus.cc:686] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [term 1 LEADER]: Becoming Leader. State: Replica: a0280de2a4c740c99ded9d1895f21d4f, State: Running, Role: LEADER
I20260811 01:49:34.329861 11556 consensus_queue.cc:237] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0280de2a4c740c99ded9d1895f21d4f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45487 } } peers { permanent_uuid: "f5f86857c9d64a299e00bc5fa8953150" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36539 } } peers { permanent_uuid: "32bd305033d540d69271ba0c770bab77" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42095 } }
I20260811 01:49:34.334477 10982 catalog_manager.cc:5495] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f reported cstate change: term changed from 0 to 1, leader changed from <none> to a0280de2a4c740c99ded9d1895f21d4f (127.21.214.67). New cstate: current_term: 1 leader_uuid: "a0280de2a4c740c99ded9d1895f21d4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0280de2a4c740c99ded9d1895f21d4f" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 45487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f86857c9d64a299e00bc5fa8953150" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32bd305033d540d69271ba0c770bab77" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42095 } health_report { overall_health: UNKNOWN } } }
I20260811 01:49:34.372107 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:34.366318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57858 (local address 127.21.214.67:45487)
0811 01:49:34.366517 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:34.366522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:34.366697 (+ 175us) server_negotiation.cc:407] Connection header received
0811 01:49:34.366884 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:34.366888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:34.366969 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:34.367099 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:49:34.367481 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.368197 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.369767 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.370293 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.370392 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:34.370958 (+ 566us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:49:34.371455 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:49:34.371607 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:34.371865 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 01:49:34.371907 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 01:49:34.380993 11147 raft_consensus.cc:1261] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150 [term 1 FOLLOWER]: Refusing update from remote peer a0280de2a4c740c99ded9d1895f21d4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:49:34.381152 11313 raft_consensus.cc:1261] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77 [term 1 FOLLOWER]: Refusing update from remote peer a0280de2a4c740c99ded9d1895f21d4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:49:34.382160 11559 consensus_queue.cc:1035] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "32bd305033d540d69271ba0c770bab77" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:49:34.382567 11556 consensus_queue.cc:1035] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f86857c9d64a299e00bc5fa8953150" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:49:34.399001 11562 mvcc.cc:205] Tried to move back new op lower bound from 7317147543053606912 to 7317147542858874880. Current Snapshot: MvccSnapshot[applied={T|T < 7317147543053606912}]
I20260811 01:49:34.401671 11565 mvcc.cc:205] Tried to move back new op lower bound from 7317147543053606912 to 7317147542858874880. Current Snapshot: MvccSnapshot[applied={T|T < 7317147543053606912}]
I20260811 01:49:34.416813 10982 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:49:34.417171 10982 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:49:34.421180 10982 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3712cb77bae64e91b6a818cb3af457a3: Sending DeleteTablet for 3 replicas of tablet c08a74710f0a4852b1b4a621a2a6107c
I20260811 01:49:34.422586 11125 tablet_service.cc:1514] Processing DeleteTablet for tablet c08a74710f0a4852b1b4a621a2a6107c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:34 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:57962
I20260811 01:49:34.422611 11288 tablet_service.cc:1514] Processing DeleteTablet for tablet c08a74710f0a4852b1b4a621a2a6107c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:34 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:36062
I20260811 01:49:34.423627 11574 tablet_replica.cc:330] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150: stopping tablet replica
I20260811 01:49:34.423682 11447 tablet_service.cc:1514] Processing DeleteTablet for tablet c08a74710f0a4852b1b4a621a2a6107c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:34 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:57856
I20260811 01:49:34.424259 11574 raft_consensus.cc:2229] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:49:34.424853 11574 raft_consensus.cc:2258] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:49:34.425750 11575 tablet_replica.cc:330] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77: stopping tablet replica
I20260811 01:49:34.425771 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 11055
I20260811 01:49:34.427850 11574 ts_tablet_manager.cc:1905] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:49:34.435117 11576 tablet_replica.cc:330] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f: stopping tablet replica
I20260811 01:49:34.435658 11576 raft_consensus.cc:2229] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:49:34.436002 11575 raft_consensus.cc:2229] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:49:34.436487 11576 raft_consensus.cc:2258] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:49:34.437186 11575 raft_consensus.cc:2258] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:49:34.438730 11574 ts_tablet_manager.cc:1918] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:49:34.438942 11574 log.cc:1196] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/c08a74710f0a4852b1b4a621a2a6107c
I20260811 01:49:34.439576 11574 ts_tablet_manager.cc:1939] T c08a74710f0a4852b1b4a621a2a6107c P f5f86857c9d64a299e00bc5fa8953150: Deleting consensus metadata
I20260811 01:49:34.440301 11576 ts_tablet_manager.cc:1905] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:49:34.440313 11575 ts_tablet_manager.cc:1905] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:49:34.442866 10964 catalog_manager.cc:4841] TS f5f86857c9d64a299e00bc5fa8953150 (127.21.214.65:36539): tablet c08a74710f0a4852b1b4a621a2a6107c (table test-table [id=06223a3b5e664dcf98dbe1b391782523]) successfully deleted
I20260811 01:49:34.448302 11576 ts_tablet_manager.cc:1918] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:49:34.448501 11576 log.cc:1196] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/c08a74710f0a4852b1b4a621a2a6107c
I20260811 01:49:34.449071 11576 ts_tablet_manager.cc:1939] T c08a74710f0a4852b1b4a621a2a6107c P a0280de2a4c740c99ded9d1895f21d4f: Deleting consensus metadata
I20260811 01:49:34.449930 11575 ts_tablet_manager.cc:1918] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:49:34.450205 11575 log.cc:1196] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/c08a74710f0a4852b1b4a621a2a6107c
I20260811 01:49:34.450842 11575 ts_tablet_manager.cc:1939] T c08a74710f0a4852b1b4a621a2a6107c P 32bd305033d540d69271ba0c770bab77: Deleting consensus metadata
Aug 11 01:49:34 dist-test-slave-9gcw krb5kdc[10930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:49:34.451169 10964 catalog_manager.cc:4841] TS a0280de2a4c740c99ded9d1895f21d4f (127.21.214.67:45487): tablet c08a74710f0a4852b1b4a621a2a6107c (table test-table [id=06223a3b5e664dcf98dbe1b391782523]) successfully deleted
I20260811 01:49:34.452781 10965 catalog_manager.cc:4841] TS 32bd305033d540d69271ba0c770bab77 (127.21.214.66:42095): tablet c08a74710f0a4852b1b4a621a2a6107c (table test-table [id=06223a3b5e664dcf98dbe1b391782523]) successfully deleted
I20260811 01:49:34.459456 11546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:34.428316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57970 (local address 127.21.214.65:36539)
0811 01:49:34.428497 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:34.428505 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:34.438477 (+ 9972us) server_negotiation.cc:407] Connection header received
0811 01:49:34.439062 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:34.439070 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:34.439160 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:34.439254 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:34.441434 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.442538 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.445379 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.446082 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.446189 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:34.452356 (+ 6167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:34.452376 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:34.452394 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:34.452450 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:34.455976 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:34.457073 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:34.457082 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:34.457084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:34.457347 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:34.457815 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:34.457818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:34.457820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:34.458206 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:34.458485 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:34.458905 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 01:49:34.459084 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 01:49:34.460172 11199 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:34.556991 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 11055
I20260811 01:49:34.586709 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 11211
Aug 11 01:49:34 dist-test-slave-9gcw krb5kdc[10930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.66@KRBTEST.COM
I20260811 01:49:34.606185 11544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:34.587642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36066 (local address 127.21.214.66:42095)
0811 01:49:34.587843 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:34.587850 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:34.587872 (+ 22us) server_negotiation.cc:407] Connection header received
0811 01:49:34.588017 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:34.588023 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:34.588113 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:34.588222 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:34.589660 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.590829 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.592474 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.593132 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.593401 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:34.598638 (+ 5237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:34.598658 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:34.598679 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:34.598729 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:34.602804 (+ 4075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:34.603647 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:34.603660 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:34.603663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:34.603827 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:34.604234 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:34.604239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:34.604244 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:34.604783 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:34.605127 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:34.605573 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 01:49:34.605810 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 01:49:34.607223 11364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:34.702951 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 11211
I20260811 01:49:34.727238 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 11377
Aug 11 01:49:34 dist-test-slave-9gcw krb5kdc[10930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.67@KRBTEST.COM
I20260811 01:49:34.747736 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:34.728246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57870 (local address 127.21.214.67:45487)
0811 01:49:34.728429 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:34.728434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:34.728453 (+ 19us) server_negotiation.cc:407] Connection header received
0811 01:49:34.728703 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:34.728708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:34.728805 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:34.728904 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:34.730848 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.731732 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.733992 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:34.734464 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:34.734576 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:34.740796 (+ 6220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:34.740809 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:34.740827 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:34.740872 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:34.744309 (+ 3437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:34.745279 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:34.745286 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:34.745288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:34.745437 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:34.745888 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:34.745891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:34.745893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:34.746280 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:34.746542 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:34.747040 (+ 498us) server_negotiation.cc:299] Negotiation successful
0811 01:49:34.747239 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 01:49:34.748739 11522 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:34.874495 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 11377
I20260811 01:49:34.898730 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 10946
I20260811 01:49:34.899928 11018 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:35.022473 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 10946
2026-08-11T01:49:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4884 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T01:49:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:49:35Z Disabled control of system clock
I20260811 01:49:35.122792 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:39697
--webserver_interface=127.21.214.126
--webserver_port=0
--builtin_ntp_servers=127.21.214.84:41651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:39697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 01:49:35.484367 11586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:35.484773 11586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:35.484863 11586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:35.492479 11586 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:49:35.492619 11586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:35.492667 11586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:35.492719 11586 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:49:35.492758 11586 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:49:35.502496 11586 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:41651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:39697
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:39697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:35.503970 11586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:35.505875 11586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:35.518919 11597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:35.518932 11593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:35.519464 11592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:35.519753 11586 server_base.cc:1034] running on GCE node
I20260811 01:49:35.520597 11586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:35.522301 11586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:35.523559 11586 hybrid_clock.cc:648] HybridClock initialized: now 1786412975523495 us; error 64 us; skew 500 ppm
I20260811 01:49:35.524013 11586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:49:35.526690 11586 webserver.cc:460] Webserver started at http://127.21.214.126:37541/ using document root <none> and password file <none>
I20260811 01:49:35.527437 11586 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:35.527577 11586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:35.527933 11586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:35.531122 11586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "38431b57224149798e74fdda8b1d4ad9"
format_stamp: "Formatted at 2026-08-11 01:49:35 on dist-test-slave-9gcw"
server_key: "d03da06da47230b1f37be6310e1c133f"
server_key_iv: "3a6483011434211f974ec13a2bcf4b10"
server_key_version: "encryptionkey@0"
I20260811 01:49:35.531996 11586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "38431b57224149798e74fdda8b1d4ad9"
format_stamp: "Formatted at 2026-08-11 01:49:35 on dist-test-slave-9gcw"
server_key: "d03da06da47230b1f37be6310e1c133f"
server_key_iv: "3a6483011434211f974ec13a2bcf4b10"
server_key_version: "encryptionkey@0"
I20260811 01:49:35.537693 11586 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 01:49:35.541563 11604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:35.543099 11586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:49:35.543311 11586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "38431b57224149798e74fdda8b1d4ad9"
format_stamp: "Formatted at 2026-08-11 01:49:35 on dist-test-slave-9gcw"
server_key: "d03da06da47230b1f37be6310e1c133f"
server_key_iv: "3a6483011434211f974ec13a2bcf4b10"
server_key_version: "encryptionkey@0"
I20260811 01:49:35.543548 11586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:35.558120 11586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:35.559029 11586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:35.559345 11586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:35.579324 11586 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:39697
I20260811 01:49:35.579370 11668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:39697 every 8 connection(s)
I20260811 01:49:35.580812 11586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:49:35.584803 11669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:35.585949 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 11586
I20260811 01:49:35.586216 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:49:35.586892 22361 external_mini_cluster.cc:1428] Setting key fa178a478e581a9bd951cc1b24363915
I20260811 01:49:35.599015 11669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9: Bootstrap starting.
I20260811 01:49:35.602840 11672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:35.588880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52752 (local address 127.21.214.126:39697)
0811 01:49:35.590278 (+ 1398us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:35.590299 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:35.590424 (+ 125us) server_negotiation.cc:407] Connection header received
0811 01:49:35.592435 (+ 2011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:35.592461 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:35.593026 (+ 565us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:35.593559 (+ 533us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:35.594258 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:35.595745 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:35.597405 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:35.598174 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:35.598399 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:35.599818 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:35.599829 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:35.599848 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:35.600579 (+ 731us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:35.600701 (+ 122us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:35.600735 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:35.600875 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:35.601122 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 01:49:35.601539 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":487,"threads_started":1}
I20260811 01:49:35.603322 11669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:35.604507 11669 log.cc:824] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:35.607547 11669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9: No bootstrap required, opened a new log
I20260811 01:49:35.615031 11669 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38431b57224149798e74fdda8b1d4ad9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 39697 } }
I20260811 01:49:35.615424 11669 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:35.615500 11669 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38431b57224149798e74fdda8b1d4ad9, State: Initialized, Role: FOLLOWER
I20260811 01:49:35.616060 11669 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38431b57224149798e74fdda8b1d4ad9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 39697 } }
I20260811 01:49:35.616285 11669 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:49:35.616389 11669 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:49:35.616520 11669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:35.619140 11669 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38431b57224149798e74fdda8b1d4ad9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 39697 } }
I20260811 01:49:35.619618 11669 leader_election.cc:302] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38431b57224149798e74fdda8b1d4ad9; no voters:
I20260811 01:49:35.620241 11669 leader_election.cc:288] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:49:35.620416 11674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:49:35.621344 11674 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9 [term 1 LEADER]: Becoming Leader. State: Replica: 38431b57224149798e74fdda8b1d4ad9, State: Running, Role: LEADER
I20260811 01:49:35.621876 11674 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38431b57224149798e74fdda8b1d4ad9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 39697 } }
I20260811 01:49:35.622649 11669 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:49:35.624974 11676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38431b57224149798e74fdda8b1d4ad9. Latest consensus state: current_term: 1 leader_uuid: "38431b57224149798e74fdda8b1d4ad9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38431b57224149798e74fdda8b1d4ad9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 39697 } } }
I20260811 01:49:35.625466 11676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9 [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:35.632006 11682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:49:35.633328 11675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38431b57224149798e74fdda8b1d4ad9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38431b57224149798e74fdda8b1d4ad9" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 39697 } } }
I20260811 01:49:35.633874 11675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9 [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:35.637543 11682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:49:35.647126 11682 catalog_manager.cc:1347] Generated new cluster ID: 9fd16b14451d47738ed2a258823a824b
I20260811 01:49:35.647264 11682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:49:35.660512 11682 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:49:35.662196 11682 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:49:35.669731 11682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9: Generated new TSK 0
I20260811 01:49:35.670717 11682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:49:35.678522 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:39697
--builtin_ntp_servers=127.21.214.84:41651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 01:49:36.052323 11695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:36.052667 11695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:36.052795 11695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:36.061281 11695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:36.061393 11695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:36.061587 11695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:49:36.071381 11695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:41651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:39697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:36.072862 11695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:36.074898 11695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:36.089800 11700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:36.089768 11705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:36.089720 11701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:36.094925 11695 server_base.cc:1034] running on GCE node
I20260811 01:49:36.095700 11695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:36.096798 11695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:36.098061 11695 hybrid_clock.cc:648] HybridClock initialized: now 1786412976098003 us; error 90 us; skew 500 ppm
I20260811 01:49:36.098425 11695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:49:36.101126 11695 webserver.cc:460] Webserver started at http://127.21.214.65:43399/ using document root <none> and password file <none>
I20260811 01:49:36.101962 11695 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:36.102098 11695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:36.102463 11695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:36.105780 11695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "2904a7220c2b46668be454991671b285"
format_stamp: "Formatted at 2026-08-11 01:49:36 on dist-test-slave-9gcw"
server_key: "89d45f4b94bfa4e465d8c017b7e488ee"
server_key_iv: "3f90bcdc26a6754ddeb064fb3ec26a13"
server_key_version: "encryptionkey@0"
I20260811 01:49:36.106618 11695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "2904a7220c2b46668be454991671b285"
format_stamp: "Formatted at 2026-08-11 01:49:36 on dist-test-slave-9gcw"
server_key: "89d45f4b94bfa4e465d8c017b7e488ee"
server_key_iv: "3f90bcdc26a6754ddeb064fb3ec26a13"
server_key_version: "encryptionkey@0"
I20260811 01:49:36.112317 11695 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 01:49:36.116268 11712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:36.118000 11695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:49:36.118221 11695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "2904a7220c2b46668be454991671b285"
format_stamp: "Formatted at 2026-08-11 01:49:36 on dist-test-slave-9gcw"
server_key: "89d45f4b94bfa4e465d8c017b7e488ee"
server_key_iv: "3f90bcdc26a6754ddeb064fb3ec26a13"
server_key_version: "encryptionkey@0"
I20260811 01:49:36.118487 11695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:36.150329 11695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:36.151314 11695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:36.151674 11695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:36.152884 11695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:36.154696 11695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:36.154881 11695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:36.155040 11695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:36.155149 11695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:36.190971 11695 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:43307
I20260811 01:49:36.191067 11841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:43307 every 8 connection(s)
I20260811 01:49:36.192420 11695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:49:36.192921 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 11695
I20260811 01:49:36.193116 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:49:36.193815 22361 external_mini_cluster.cc:1428] Setting key a3fe7561be958ece4ff2ea3d9dcea2c4
I20260811 01:49:36.197357 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:39697
--builtin_ntp_servers=127.21.214.84:41651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 01:49:36.207598 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:36.195380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:51171 (local address 127.21.214.126:39697)
0811 01:49:36.195904 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:36.195910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:36.197439 (+ 1529us) server_negotiation.cc:407] Connection header received
0811 01:49:36.199649 (+ 2210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:36.199656 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:36.199782 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:36.199929 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:36.201254 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:36.202112 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:36.204003 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:36.204493 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:36.204715 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:36.206472 (+ 1757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:36.206478 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:36.206482 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:36.206505 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:36.206555 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:36.206571 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:36.206707 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:36.207000 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 01:49:36.207138 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":302,"threads_started":1}
I20260811 01:49:36.208909 11845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:36.196030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:39697 (local address 127.21.214.65:51171)
0811 01:49:36.197259 (+ 1229us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:36.197315 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:36.199313 (+ 1998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:36.200228 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:36.200237 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:36.200337 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:49:36.200999 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:36.201028 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:36.202302 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:36.202309 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:36.203820 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:36.203834 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:36.204837 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:36.204845 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:36.205132 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:36.206073 (+ 941us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:49:36.206076 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:49:36.206138 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:49:36.206228 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:49:36.206705 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:49:36.206721 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:49:36.206733 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 01:49:36.206965 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 01:49:36.207428 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":392,"threads_started":1}
I20260811 01:49:36.211189 11842 heartbeater.cc:347] Connected to a master server at 127.21.214.126:39697
I20260811 01:49:36.211589 11842 heartbeater.cc:464] Registering TS with master...
I20260811 01:49:36.212527 11842 heartbeater.cc:511] Master 127.21.214.126:39697 requested a full tablet report, sending...
I20260811 01:49:36.215135 11625 ts_manager.cc:194] Registered new tserver with Master: 2904a7220c2b46668be454991671b285 (127.21.214.65:43307)
I20260811 01:49:36.216909 11625 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.214.65:51171
W20260811 01:49:36.551766 11848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:36.552136 11848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:36.552322 11848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:36.559844 11848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:36.559999 11848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:36.560181 11848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:49:36.569429 11848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:41651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:39697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:36.570853 11848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:36.572748 11848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:36.589211 11855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:36.589232 11854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:36.589709 11857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:36.590612 11848 server_base.cc:1034] running on GCE node
I20260811 01:49:36.591300 11848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:36.592365 11848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:36.593598 11848 hybrid_clock.cc:648] HybridClock initialized: now 1786412976593543 us; error 47 us; skew 500 ppm
I20260811 01:49:36.593940 11848 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:49:36.596364 11848 webserver.cc:460] Webserver started at http://127.21.214.66:46115/ using document root <none> and password file <none>
I20260811 01:49:36.597085 11848 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:36.597250 11848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:36.597573 11848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:36.600528 11848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "d2b079fb31fe45d4a52b126198076845"
format_stamp: "Formatted at 2026-08-11 01:49:36 on dist-test-slave-9gcw"
server_key: "22920de9c6fe5a66f981a0abc24a8334"
server_key_iv: "7e7585960ed52c9c4d47860f761eba44"
server_key_version: "encryptionkey@0"
I20260811 01:49:36.601411 11848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "d2b079fb31fe45d4a52b126198076845"
format_stamp: "Formatted at 2026-08-11 01:49:36 on dist-test-slave-9gcw"
server_key: "22920de9c6fe5a66f981a0abc24a8334"
server_key_iv: "7e7585960ed52c9c4d47860f761eba44"
server_key_version: "encryptionkey@0"
I20260811 01:49:36.606595 11848 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 01:49:36.610108 11864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:36.611610 11848 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 01:49:36.611785 11848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "d2b079fb31fe45d4a52b126198076845"
format_stamp: "Formatted at 2026-08-11 01:49:36 on dist-test-slave-9gcw"
server_key: "22920de9c6fe5a66f981a0abc24a8334"
server_key_iv: "7e7585960ed52c9c4d47860f761eba44"
server_key_version: "encryptionkey@0"
I20260811 01:49:36.612021 11848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:36.647251 11848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:36.648193 11848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:36.648526 11848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:36.649610 11848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:36.651322 11848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:36.651410 11848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:36.651566 11848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:36.651640 11848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:36.688170 11848 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:36701
I20260811 01:49:36.688271 12000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:36701 every 8 connection(s)
I20260811 01:49:36.689824 11848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:49:36.691844 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 11848
I20260811 01:49:36.692034 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:49:36.692569 22361 external_mini_cluster.cc:1428] Setting key 08b827c3ecd4704cd3ab8a81e860a91e
I20260811 01:49:36.696213 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:39697
--builtin_ntp_servers=127.21.214.84:41651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 01:49:36.706455 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:36.692898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:37689 (local address 127.21.214.126:39697)
0811 01:49:36.693185 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:36.693195 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:36.695198 (+ 2003us) server_negotiation.cc:407] Connection header received
0811 01:49:36.698134 (+ 2936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:36.698139 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:36.698209 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:36.698309 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:36.699879 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:36.700765 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:36.703372 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:36.703816 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:36.703927 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:36.705697 (+ 1770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:36.705702 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:36.705706 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:36.705726 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:36.705759 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:36.705772 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:36.705847 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:36.706123 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 01:49:36.706206 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 01:49:36.708032 12004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:36.693722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:39697 (local address 127.21.214.66:37689)
0811 01:49:36.695007 (+ 1285us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:36.695057 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:36.697767 (+ 2710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:36.698607 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:36.698621 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:36.698758 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:49:36.699657 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:36.699696 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:36.700940 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:36.700948 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:36.703198 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:36.703216 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:36.704062 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:36.704072 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:36.704421 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:36.705307 (+ 886us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:49:36.705312 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:49:36.705369 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:49:36.705488 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:49:36.705877 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:49:36.705890 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:49:36.705902 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 01:49:36.706091 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 01:49:36.706538 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":357,"threads_started":1}
I20260811 01:49:36.709615 12001 heartbeater.cc:347] Connected to a master server at 127.21.214.126:39697
I20260811 01:49:36.709975 12001 heartbeater.cc:464] Registering TS with master...
I20260811 01:49:36.710951 12001 heartbeater.cc:511] Master 127.21.214.126:39697 requested a full tablet report, sending...
I20260811 01:49:36.712612 11625 ts_manager.cc:194] Registered new tserver with Master: d2b079fb31fe45d4a52b126198076845 (127.21.214.66:36701)
I20260811 01:49:36.713526 11625 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.214.66:37689
W20260811 01:49:37.061481 12005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:37.061787 12005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:37.061969 12005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:37.069577 12005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:37.069701 12005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:37.069875 12005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:49:37.079401 12005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:41651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:39697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:37.081073 12005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:37.082984 12005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:37.097343 12012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:37.097277 12011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:37.097930 12014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:37.098465 12005 server_base.cc:1034] running on GCE node
I20260811 01:49:37.099128 12005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:37.100152 12005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:37.101356 12005 hybrid_clock.cc:648] HybridClock initialized: now 1786412977101282 us; error 63 us; skew 500 ppm
I20260811 01:49:37.101744 12005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:49:37.104125 12005 webserver.cc:460] Webserver started at http://127.21.214.67:40969/ using document root <none> and password file <none>
I20260811 01:49:37.104892 12005 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:37.105023 12005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:37.105435 12005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:37.108722 12005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "18e03e9e7bb342c791cdf13c89329a75"
format_stamp: "Formatted at 2026-08-11 01:49:37 on dist-test-slave-9gcw"
server_key: "85873fba8158fc4318e61e36bbb64415"
server_key_iv: "3617abe7cea1e9edb965e14b4dec6ced"
server_key_version: "encryptionkey@0"
I20260811 01:49:37.109809 12005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "18e03e9e7bb342c791cdf13c89329a75"
format_stamp: "Formatted at 2026-08-11 01:49:37 on dist-test-slave-9gcw"
server_key: "85873fba8158fc4318e61e36bbb64415"
server_key_iv: "3617abe7cea1e9edb965e14b4dec6ced"
server_key_version: "encryptionkey@0"
I20260811 01:49:37.115679 12005 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 01:49:37.119809 12023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:37.121378 12005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 01:49:37.121573 12005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "18e03e9e7bb342c791cdf13c89329a75"
format_stamp: "Formatted at 2026-08-11 01:49:37 on dist-test-slave-9gcw"
server_key: "85873fba8158fc4318e61e36bbb64415"
server_key_iv: "3617abe7cea1e9edb965e14b4dec6ced"
server_key_version: "encryptionkey@0"
I20260811 01:49:37.121835 12005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:37.160830 12005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:37.161823 12005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:37.162164 12005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:37.163223 12005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:37.164810 12005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:37.164938 12005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:37.165084 12005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:37.165202 12005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:37.200557 12005 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:41695
I20260811 01:49:37.200616 12161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:41695 every 8 connection(s)
I20260811 01:49:37.201985 12005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:49:37.203296 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 12005
I20260811 01:49:37.203490 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:49:37.203946 22361 external_mini_cluster.cc:1428] Setting key afad1590ab72d66932cc341c919c6e3f
I20260811 01:49:37.216266 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.204892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:45165 (local address 127.21.214.126:39697)
0811 01:49:37.205099 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:37.205104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:37.207435 (+ 2331us) server_negotiation.cc:407] Connection header received
0811 01:49:37.209362 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:37.209367 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:37.209445 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:37.209547 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:37.210767 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.211618 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.213280 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.213815 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.213929 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:37.215406 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:37.215411 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:37.215415 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:37.215435 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:37.215492 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:37.215507 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:37.215588 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:37.215828 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 01:49:37.215924 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 01:49:37.217455 12165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.205756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:39697 (local address 127.21.214.67:45165)
0811 01:49:37.207267 (+ 1511us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:37.207315 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:37.209016 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:37.209782 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:37.209791 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:37.209905 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:49:37.210536 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:37.210554 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.211768 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.211771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:37.213056 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:37.213068 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.214054 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.214059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:37.214302 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:37.215058 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:49:37.215061 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:49:37.215102 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:49:37.215203 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:49:37.215587 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:49:37.215596 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:49:37.215602 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 01:49:37.215763 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 01:49:37.216106 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":932,"thread_start_us":695,"threads_started":1}
I20260811 01:49:37.219058 12162 heartbeater.cc:347] Connected to a master server at 127.21.214.126:39697
I20260811 01:49:37.219453 12162 heartbeater.cc:464] Registering TS with master...
I20260811 01:49:37.220396 12162 heartbeater.cc:511] Master 127.21.214.126:39697 requested a full tablet report, sending...
I20260811 01:49:37.220425 11842 heartbeater.cc:503] Master 127.21.214.126:39697 was elected leader, sending a full tablet report...
I20260811 01:49:37.221864 11625 ts_manager.cc:194] Registered new tserver with Master: 18e03e9e7bb342c791cdf13c89329a75 (127.21.214.67:41695)
I20260811 01:49:37.222743 11625 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.214.67:45165
I20260811 01:49:37.233423 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:49:37.243420 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.237670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52760 (local address 127.21.214.126:39697)
0811 01:49:37.237882 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:37.237887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:37.238309 (+ 422us) server_negotiation.cc:407] Connection header received
0811 01:49:37.238446 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:37.238450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:37.238512 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:37.238603 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:37.239080 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.239788 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.241127 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.241613 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.241711 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:37.242777 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:37.242784 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:37.242788 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:37.242807 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:37.242839 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:37.242851 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:37.242967 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:37.243127 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 01:49:37.243174 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 01:49:37.247498 11625 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:49:37.254999 11625 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:49:37.283691 12175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.269428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:41695 (local address 127.0.0.1:46702)
0811 01:49:37.271118 (+ 1690us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:37.271168 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:37.271447 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:37.273540 (+ 2093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:37.273549 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:37.273589 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:37.274185 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:37.274203 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.277488 (+ 3285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.277494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:37.280242 (+ 2748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:37.280253 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.282369 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.282377 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:37.282665 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:37.282879 (+ 214us) client_negotiation.cc:769] Sending connection context
0811 01:49:37.283033 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 01:49:37.283263 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1344,"mutex_wait_us":86,"thread_start_us":1165,"threads_started":1}
I20260811 01:49:37.283727 12178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.270182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46702 (local address 127.21.214.67:41695)
0811 01:49:37.272755 (+ 2573us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:37.272766 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:37.272807 (+ 41us) server_negotiation.cc:407] Connection header received
0811 01:49:37.272917 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:37.272925 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:37.273256 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:37.274486 (+ 1230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:37.274866 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.276718 (+ 1852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.280440 (+ 3722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.282064 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.282252 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:37.282588 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:37.283105 (+ 517us) server_negotiation.cc:299] Negotiation successful
0811 01:49:37.283328 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1822,"thread_start_us":1643,"threads_started":1}
I20260811 01:49:37.287318 12180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.269487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:43307 (local address 127.0.0.1:39160)
0811 01:49:37.278350 (+ 8863us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:37.278377 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:37.278552 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:37.279405 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:37.279410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:37.279437 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:37.279925 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:37.279939 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.281957 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.281963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:37.284391 (+ 2428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:37.284404 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.286420 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.286428 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:37.286602 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:37.286635 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 01:49:37.286778 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 01:49:37.286928 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8663,"mutex_wait_us":63,"thread_start_us":8933,"threads_started":1}
I20260811 01:49:37.287388 12177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.269646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39160 (local address 127.21.214.65:43307)
0811 01:49:37.277356 (+ 7710us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:37.277365 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:37.278687 (+ 1322us) server_negotiation.cc:407] Connection header received
0811 01:49:37.278832 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:37.278839 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:37.279155 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:37.279406 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:37.280123 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.281791 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.284598 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.286198 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.286363 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:37.286657 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:37.286831 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 01:49:37.287008 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7469,"thread_start_us":7325,"threads_started":1}
I20260811 01:49:37.287938 12182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.271774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51440 (local address 127.21.214.66:36701)
0811 01:49:37.277716 (+ 5942us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:37.277728 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:37.278858 (+ 1130us) server_negotiation.cc:407] Connection header received
0811 01:49:37.279034 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:37.279040 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:37.279243 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:37.279412 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:37.280474 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.282013 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.284857 (+ 2844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.286259 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.286395 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:37.286629 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:37.287418 (+ 789us) server_negotiation.cc:299] Negotiation successful
0811 01:49:37.287606 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5776,"thread_start_us":5657,"threads_started":1}
I20260811 01:49:37.288399 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.270959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:36701 (local address 127.0.0.1:51440)
0811 01:49:37.278706 (+ 7747us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:37.278725 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:37.278869 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:37.279677 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:37.279681 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:37.279705 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:37.280226 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:37.280242 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.282175 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.282182 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:37.284629 (+ 2447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:37.284660 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.286770 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.286776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:37.286957 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:37.286990 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 01:49:37.287080 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 01:49:37.287218 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7625,"mutex_wait_us":40,"thread_start_us":6975,"threads_started":1}
I20260811 01:49:37.288452 12084 tablet_service.cc:1467] Processing CreateTablet for tablet 362d543c3d7a4c2d911b9172c49f7d29 (DEFAULT_TABLE table=test-table [id=5bacd6325b5f4e8394a4750aeb08d774]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:49:37.290941 12084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 362d543c3d7a4c2d911b9172c49f7d29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:37.291478 11766 tablet_service.cc:1467] Processing CreateTablet for tablet 362d543c3d7a4c2d911b9172c49f7d29 (DEFAULT_TABLE table=test-table [id=5bacd6325b5f4e8394a4750aeb08d774]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:49:37.291901 11927 tablet_service.cc:1467] Processing CreateTablet for tablet 362d543c3d7a4c2d911b9172c49f7d29 (DEFAULT_TABLE table=test-table [id=5bacd6325b5f4e8394a4750aeb08d774]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:49:37.293021 11927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 362d543c3d7a4c2d911b9172c49f7d29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:37.293051 11766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 362d543c3d7a4c2d911b9172c49f7d29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:37.305546 12183 tablet_bootstrap.cc:492] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845: Bootstrap starting.
I20260811 01:49:37.308403 12186 tablet_bootstrap.cc:492] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285: Bootstrap starting.
I20260811 01:49:37.309888 12183 tablet_bootstrap.cc:654] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:37.310415 12184 tablet_bootstrap.cc:492] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75: Bootstrap starting.
I20260811 01:49:37.311457 12183 log.cc:824] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:37.312309 12186 tablet_bootstrap.cc:654] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:37.313629 12186 log.cc:824] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:37.314357 12184 tablet_bootstrap.cc:654] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:37.315616 12183 tablet_bootstrap.cc:492] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845: No bootstrap required, opened a new log
I20260811 01:49:37.315651 12184 log.cc:824] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:37.315915 12183 ts_tablet_manager.cc:1397] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 01:49:37.317579 12186 tablet_bootstrap.cc:492] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285: No bootstrap required, opened a new log
I20260811 01:49:37.317866 12186 ts_tablet_manager.cc:1397] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260811 01:49:37.318578 12184 tablet_bootstrap.cc:492] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75: No bootstrap required, opened a new log
I20260811 01:49:37.318848 12184 ts_tablet_manager.cc:1397] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 01:49:37.327477 12184 raft_consensus.cc:348] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2904a7220c2b46668be454991671b285" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 43307 } } peers { permanent_uuid: "18e03e9e7bb342c791cdf13c89329a75" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 41695 } } peers { permanent_uuid: "d2b079fb31fe45d4a52b126198076845" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 36701 } }
I20260811 01:49:37.327975 12184 raft_consensus.cc:374] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:37.328075 12184 raft_consensus.cc:729] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e03e9e7bb342c791cdf13c89329a75, State: Initialized, Role: FOLLOWER
I20260811 01:49:37.327950 12183 raft_consensus.cc:348] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2904a7220c2b46668be454991671b285" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 43307 } } peers { permanent_uuid: "18e03e9e7bb342c791cdf13c89329a75" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 41695 } } peers { permanent_uuid: "d2b079fb31fe45d4a52b126198076845" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 36701 } }
I20260811 01:49:37.328423 12183 raft_consensus.cc:374] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:37.328542 12183 raft_consensus.cc:729] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2b079fb31fe45d4a52b126198076845, State: Initialized, Role: FOLLOWER
I20260811 01:49:37.328718 12186 raft_consensus.cc:348] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2904a7220c2b46668be454991671b285" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 43307 } } peers { permanent_uuid: "18e03e9e7bb342c791cdf13c89329a75" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 41695 } } peers { permanent_uuid: "d2b079fb31fe45d4a52b126198076845" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 36701 } }
I20260811 01:49:37.328979 12184 consensus_queue.cc:260] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2904a7220c2b46668be454991671b285" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 43307 } } peers { permanent_uuid: "18e03e9e7bb342c791cdf13c89329a75" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 41695 } } peers { permanent_uuid: "d2b079fb31fe45d4a52b126198076845" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 36701 } }
I20260811 01:49:37.329238 12186 raft_consensus.cc:374] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:37.329336 12186 raft_consensus.cc:729] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2904a7220c2b46668be454991671b285, State: Initialized, Role: FOLLOWER
I20260811 01:49:37.329216 12183 consensus_queue.cc:260] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2904a7220c2b46668be454991671b285" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 43307 } } peers { permanent_uuid: "18e03e9e7bb342c791cdf13c89329a75" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 41695 } } peers { permanent_uuid: "d2b079fb31fe45d4a52b126198076845" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 36701 } }
I20260811 01:49:37.330071 12186 consensus_queue.cc:260] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2904a7220c2b46668be454991671b285" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 43307 } } peers { permanent_uuid: "18e03e9e7bb342c791cdf13c89329a75" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 41695 } } peers { permanent_uuid: "d2b079fb31fe45d4a52b126198076845" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 36701 } }
I20260811 01:49:37.332495 12186 ts_tablet_manager.cc:1428] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260811 01:49:37.333583 12162 heartbeater.cc:503] Master 127.21.214.126:39697 was elected leader, sending a full tablet report...
I20260811 01:49:37.334229 12184 ts_tablet_manager.cc:1428] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260811 01:49:37.337886 12001 heartbeater.cc:503] Master 127.21.214.126:39697 was elected leader, sending a full tablet report...
I20260811 01:49:37.339869 12183 ts_tablet_manager.cc:1428] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845: Time spent starting tablet: real 0.024s user 0.018s sys 0.004s
I20260811 01:49:37.428996 12192 raft_consensus.cc:482] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:49:37.429373 12192 raft_consensus.cc:504] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2904a7220c2b46668be454991671b285" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 43307 } } peers { permanent_uuid: "18e03e9e7bb342c791cdf13c89329a75" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 41695 } } peers { permanent_uuid: "d2b079fb31fe45d4a52b126198076845" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 36701 } }
I20260811 01:49:37.431376 12192 leader_election.cc:288] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2904a7220c2b46668be454991671b285 (127.21.214.65:43307), d2b079fb31fe45d4a52b126198076845 (127.21.214.66:36701)
I20260811 01:49:37.437668 12177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.431815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:48159 (local address 127.21.214.65:43307)
0811 01:49:37.432074 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:37.432085 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:37.432328 (+ 243us) server_negotiation.cc:407] Connection header received
0811 01:49:37.432522 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:37.432527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:37.432611 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:37.432720 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:37.433200 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.433912 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.435815 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.436690 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.436887 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:37.436961 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:37.437258 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 01:49:37.437391 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 01:49:37.437700 12165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.431763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:43307 (local address 127.21.214.67:48159)
0811 01:49:37.432138 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:37.432180 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:37.432337 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:37.432797 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:37.432804 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:37.432832 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:37.433004 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:37.433013 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.434061 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.434066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:37.435633 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:37.435645 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.436870 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.436876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:37.437044 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:37.437067 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 01:49:37.437227 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 01:49:37.437391 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":50}
I20260811 01:49:37.438464 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.431803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:36701 (local address 127.21.214.67:38983)
0811 01:49:37.432775 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:37.432791 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:37.432981 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:37.433386 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:37.433390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:37.433413 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:37.433746 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:37.433755 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.434763 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.434767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:37.436394 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:37.436407 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.437637 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.437643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:37.437877 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:37.437915 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 01:49:37.438009 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 01:49:37.438138 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":863,"mutex_wait_us":23,"thread_start_us":555,"threads_started":1}
I20260811 01:49:37.438675 12182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.432050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:38983 (local address 127.21.214.66:36701)
0811 01:49:37.432238 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:37.432243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:37.432918 (+ 675us) server_negotiation.cc:407] Connection header received
0811 01:49:37.433130 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:37.433184 (+ 54us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:37.433268 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:37.433369 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:37.433912 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.434589 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.436578 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.437460 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.437631 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:37.437700 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:37.438320 (+ 620us) server_negotiation.cc:299] Negotiation successful
0811 01:49:37.438423 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 01:49:37.438903 11787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362d543c3d7a4c2d911b9172c49f7d29" candidate_uuid: "18e03e9e7bb342c791cdf13c89329a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2904a7220c2b46668be454991671b285" is_pre_election: true
I20260811 01:49:37.439601 11787 raft_consensus.cc:2454] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18e03e9e7bb342c791cdf13c89329a75 in term 0.
I20260811 01:49:37.439662 11949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362d543c3d7a4c2d911b9172c49f7d29" candidate_uuid: "18e03e9e7bb342c791cdf13c89329a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2b079fb31fe45d4a52b126198076845" is_pre_election: true
I20260811 01:49:37.440197 11949 raft_consensus.cc:2454] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18e03e9e7bb342c791cdf13c89329a75 in term 0.
I20260811 01:49:37.440335 12025 leader_election.cc:302] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e03e9e7bb342c791cdf13c89329a75, 2904a7220c2b46668be454991671b285; no voters:
I20260811 01:49:37.440831 12192 raft_consensus.cc:2781] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:49:37.441002 12192 raft_consensus.cc:482] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:49:37.441133 12192 raft_consensus.cc:3037] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [term 0 FOLLOWER]: Advancing to term 1
W20260811 01:49:37.442045 12002 tablet.cc:2369] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:49:37.443552 12192 raft_consensus.cc:504] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2904a7220c2b46668be454991671b285" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 43307 } } peers { permanent_uuid: "18e03e9e7bb342c791cdf13c89329a75" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 41695 } } peers { permanent_uuid: "d2b079fb31fe45d4a52b126198076845" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 36701 } }
I20260811 01:49:37.444175 12192 leader_election.cc:288] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [CANDIDATE]: Term 1 election: Requested vote from peers 2904a7220c2b46668be454991671b285 (127.21.214.65:43307), d2b079fb31fe45d4a52b126198076845 (127.21.214.66:36701)
I20260811 01:49:37.444742 11787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362d543c3d7a4c2d911b9172c49f7d29" candidate_uuid: "18e03e9e7bb342c791cdf13c89329a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2904a7220c2b46668be454991671b285"
I20260811 01:49:37.444739 11949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "362d543c3d7a4c2d911b9172c49f7d29" candidate_uuid: "18e03e9e7bb342c791cdf13c89329a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2b079fb31fe45d4a52b126198076845"
I20260811 01:49:37.444967 11949 raft_consensus.cc:3037] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:37.444976 11787 raft_consensus.cc:3037] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285 [term 0 FOLLOWER]: Advancing to term 1
W20260811 01:49:37.445132 11843 tablet.cc:2369] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:49:37.448408 11787 raft_consensus.cc:2454] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e03e9e7bb342c791cdf13c89329a75 in term 1.
I20260811 01:49:37.448407 11949 raft_consensus.cc:2454] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e03e9e7bb342c791cdf13c89329a75 in term 1.
I20260811 01:49:37.448922 12025 leader_election.cc:302] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e03e9e7bb342c791cdf13c89329a75, 2904a7220c2b46668be454991671b285; no voters:
I20260811 01:49:37.449306 12192 raft_consensus.cc:2781] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:49:37.449790 12192 raft_consensus.cc:686] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [term 1 LEADER]: Becoming Leader. State: Replica: 18e03e9e7bb342c791cdf13c89329a75, State: Running, Role: LEADER
I20260811 01:49:37.450294 12192 consensus_queue.cc:237] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2904a7220c2b46668be454991671b285" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 43307 } } peers { permanent_uuid: "18e03e9e7bb342c791cdf13c89329a75" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 41695 } } peers { permanent_uuid: "d2b079fb31fe45d4a52b126198076845" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 36701 } }
W20260811 01:49:37.453635 12163 tablet.cc:2369] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:49:37.455354 11625 catalog_manager.cc:5495] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18e03e9e7bb342c791cdf13c89329a75 (127.21.214.67). New cstate: current_term: 1 leader_uuid: "18e03e9e7bb342c791cdf13c89329a75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2904a7220c2b46668be454991671b285" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 43307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e03e9e7bb342c791cdf13c89329a75" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 41695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2b079fb31fe45d4a52b126198076845" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 36701 } health_report { overall_health: UNKNOWN } } }
I20260811 01:49:37.480044 12178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.474377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46716 (local address 127.21.214.67:41695)
0811 01:49:37.474553 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:37.474558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:37.474577 (+ 19us) server_negotiation.cc:407] Connection header received
0811 01:49:37.474695 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:37.474699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:37.474791 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:37.474967 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:49:37.475441 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.476255 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.477719 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.478170 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.478273 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:37.478878 (+ 605us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:49:37.479436 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:49:37.479594 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:37.479791 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 01:49:37.479832 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 01:49:37.489107 11949 raft_consensus.cc:1261] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845 [term 1 FOLLOWER]: Refusing update from remote peer 18e03e9e7bb342c791cdf13c89329a75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:49:37.489104 11787 raft_consensus.cc:1261] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285 [term 1 FOLLOWER]: Refusing update from remote peer 18e03e9e7bb342c791cdf13c89329a75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:49:37.490483 12192 consensus_queue.cc:1035] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2b079fb31fe45d4a52b126198076845" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 36701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:49:37.490852 12197 consensus_queue.cc:1035] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2904a7220c2b46668be454991671b285" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 43307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:49:37.505429 12202 mvcc.cc:205] Tried to move back new op lower bound from 7317147555783176192 to 7317147555640147968. Current Snapshot: MvccSnapshot[applied={T|T < 7317147555783176192}]
I20260811 01:49:37.519752 11623 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:49:37.520097 11623 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:49:37.524555 11623 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 38431b57224149798e74fdda8b1d4ad9: Sending DeleteTablet for 3 replicas of tablet 362d543c3d7a4c2d911b9172c49f7d29
I20260811 01:49:37.525931 11766 tablet_service.cc:1514] Processing DeleteTablet for tablet 362d543c3d7a4c2d911b9172c49f7d29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:37 UTC) from {username='slave'} at 127.0.0.1:39160
I20260811 01:49:37.525945 12084 tablet_service.cc:1514] Processing DeleteTablet for tablet 362d543c3d7a4c2d911b9172c49f7d29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:37 UTC) from {username='slave'} at 127.0.0.1:46702
I20260811 01:49:37.526793 11927 tablet_service.cc:1514] Processing DeleteTablet for tablet 362d543c3d7a4c2d911b9172c49f7d29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:37 UTC) from {username='slave'} at 127.0.0.1:51440
I20260811 01:49:37.527189 12212 tablet_replica.cc:330] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75: stopping tablet replica
I20260811 01:49:37.527819 12212 raft_consensus.cc:2229] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:49:37.528422 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 11695
I20260811 01:49:37.528861 12212 raft_consensus.cc:2258] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:49:37.532047 12212 ts_tablet_manager.cc:1905] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:49:37.543082 12212 ts_tablet_manager.cc:1918] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:49:37.543470 12212 log.cc:1196] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/362d543c3d7a4c2d911b9172c49f7d29
I20260811 01:49:37.544389 12212 ts_tablet_manager.cc:1939] T 362d543c3d7a4c2d911b9172c49f7d29 P 18e03e9e7bb342c791cdf13c89329a75: Deleting consensus metadata
I20260811 01:49:37.547219 11609 catalog_manager.cc:4841] TS 18e03e9e7bb342c791cdf13c89329a75 (127.21.214.67:41695): tablet 362d543c3d7a4c2d911b9172c49f7d29 (table test-table [id=5bacd6325b5f4e8394a4750aeb08d774]) successfully deleted
I20260811 01:49:37.554831 12214 tablet_replica.cc:330] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845: stopping tablet replica
I20260811 01:49:37.555622 12214 raft_consensus.cc:2229] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:49:37.556439 12214 raft_consensus.cc:2258] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:49:37.557487 12211 tablet_replica.cc:330] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285: stopping tablet replica
I20260811 01:49:37.557964 12211 raft_consensus.cc:2229] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:49:37.558537 12211 raft_consensus.cc:2258] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:49:37.559146 12214 ts_tablet_manager.cc:1905] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:49:37.560951 12211 ts_tablet_manager.cc:1905] T 362d543c3d7a4c2d911b9172c49f7d29 P 2904a7220c2b46668be454991671b285: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:49:37.562695 12177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.530071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39176 (local address 127.21.214.65:43307)
0811 01:49:37.530260 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:37.530267 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:37.553940 (+ 23673us) server_negotiation.cc:407] Connection header received
0811 01:49:37.554189 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:37.554195 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:37.554353 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:37.554475 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:37.555187 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.556451 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.558397 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.559087 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.559197 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:37.560845 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:37.560860 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:37.560878 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:37.561738 (+ 860us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:37.561812 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:37.561839 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:37.561938 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:37.562200 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 01:49:37.562325 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 01:49:37.564003 11835 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:37.567469 12214 ts_tablet_manager.cc:1918] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:49:37.567926 12214 log.cc:1196] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/362d543c3d7a4c2d911b9172c49f7d29
I20260811 01:49:37.568903 12214 ts_tablet_manager.cc:1939] T 362d543c3d7a4c2d911b9172c49f7d29 P d2b079fb31fe45d4a52b126198076845: Deleting consensus metadata
I20260811 01:49:37.570909 11606 catalog_manager.cc:4841] TS d2b079fb31fe45d4a52b126198076845 (127.21.214.66:36701): tablet 362d543c3d7a4c2d911b9172c49f7d29 (table test-table [id=5bacd6325b5f4e8394a4750aeb08d774]) successfully deleted
I20260811 01:49:37.711397 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 11695
W20260811 01:49:37.739681 11605 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.214.65:43307 (error 108)
I20260811 01:49:37.739970 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 11848
I20260811 01:49:37.742355 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.741824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:43307 (local address 127.0.0.1:39178)
0811 01:49:37.742034 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:37.742172 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:43307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260811 01:49:37.742633 11605 catalog_manager.cc:4584] TS 2904a7220c2b46668be454991671b285 (127.21.214.65:43307): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 362d543c3d7a4c2d911b9172c49f7d29: Network error: Client connection negotiation failed: client connection to 127.21.214.65:43307: connect: Connection refused (error 111)
I20260811 01:49:37.749224 12182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.741396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51446 (local address 127.21.214.66:36701)
0811 01:49:37.741621 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:37.741630 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:37.741655 (+ 25us) server_negotiation.cc:407] Connection header received
0811 01:49:37.741735 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:37.741741 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:37.741890 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:37.742012 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:37.742622 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.743550 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.745582 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.746023 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.746132 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:37.747441 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:37.747457 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:37.747485 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:37.748247 (+ 762us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:37.748303 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:37.748328 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:37.748438 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:37.748664 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 01:49:37.748793 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 01:49:37.750067 11996 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:37.796648 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.796085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:43307 (local address 127.0.0.1:39186)
0811 01:49:37.796354 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:37.796468 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:43307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 01:49:37.797972 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.797493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:43307 (local address 127.0.0.1:39188)
0811 01:49:37.797683 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:37.797794 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:43307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 01:49:37.841899 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 11848
I20260811 01:49:37.858614 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.858107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:43307 (local address 127.0.0.1:39200)
0811 01:49:37.858349 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:37.858438 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:43307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 01:49:37.859740 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.859382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:43307 (local address 127.0.0.1:39214)
0811 01:49:37.859536 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:37.859615 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:43307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 01:49:37.872434 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 12005
I20260811 01:49:37.884753 12178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.873410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46720 (local address 127.21.214.67:41695)
0811 01:49:37.873630 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:37.873637 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:37.873668 (+ 31us) server_negotiation.cc:407] Connection header received
0811 01:49:37.873863 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:37.873869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:37.874013 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:37.874156 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:37.874583 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.875813 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.877482 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:37.878173 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:37.878331 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:37.882761 (+ 4430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:37.882789 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:37.882816 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:37.883764 (+ 948us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:37.883855 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:37.883888 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:37.884027 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:37.884185 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 01:49:37.884318 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 01:49:37.885730 12157 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:37.940794 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.940235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:43307 (local address 127.0.0.1:39220)
0811 01:49:37.940485 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:37.940592 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:43307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 01:49:37.942188 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:37.941623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:43307 (local address 127.0.0.1:39230)
0811 01:49:37.941976 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:37.942057 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:43307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250}
I20260811 01:49:38.014736 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 12005
I20260811 01:49:38.039619 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 11586
I20260811 01:49:38.040830 11662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:38.154031 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:38.153415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:43307 (local address 127.0.0.1:39246)
0811 01:49:38.153604 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:38.153719 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.65:43307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 01:49:38.154323 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 11586
2026-08-11T01:49:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3130 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:49:38 dist-test-slave-9gcw krb5kdc[12228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:49:38 dist-test-slave-9gcw krb5kdc[12228](info): set up 2 sockets
Aug 11 01:49:38 dist-test-slave-9gcw krb5kdc[12228](info): commencing operation
krb5kdc: starting...
W20260811 01:49:39.400357 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.003s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:49:39 dist-test-slave-9gcw krb5kdc[12228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:49:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:49:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:49:39.737779 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:35817
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:37421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:35817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:49:40.098853 12244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:40.099174 12244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:40.099359 12244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:40.107033 12244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:49:40.107165 12244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:40.107259 12244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:40.107362 12244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:49:40.107445 12244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:49:40.117707 12244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:37421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:35817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:35817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:40.119179 12244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:40.120661 12244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:40.134136 12249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:40.134104 12250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:40.134191 12252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:40.135245 12244 server_base.cc:1034] running on GCE node
I20260811 01:49:40.136179 12244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:40.137957 12244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:40.139240 12244 hybrid_clock.cc:648] HybridClock initialized: now 1786412980139180 us; error 62 us; skew 500 ppm
I20260811 01:49:40.139635 12244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:40 dist-test-slave-9gcw krb5kdc[12228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412980, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:40.143462 12244 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:49:40.145293 12244 webserver.cc:460] Webserver started at http://127.21.214.126:38967/ using document root <none> and password file <none>
I20260811 01:49:40.146101 12244 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:40.146267 12244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:40.146621 12244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:40.149945 12244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "1e6fa03ce9e94e63a2f4a2be663033b7"
format_stamp: "Formatted at 2026-08-11 01:49:40 on dist-test-slave-9gcw"
server_key: "ebdde48e4d3f68fd74fa0988ac75fbb3"
server_key_iv: "554d9ff1dcc01c86f886e64d521ba927"
server_key_version: "encryptionkey@0"
I20260811 01:49:40.150795 12244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "1e6fa03ce9e94e63a2f4a2be663033b7"
format_stamp: "Formatted at 2026-08-11 01:49:40 on dist-test-slave-9gcw"
server_key: "ebdde48e4d3f68fd74fa0988ac75fbb3"
server_key_iv: "554d9ff1dcc01c86f886e64d521ba927"
server_key_version: "encryptionkey@0"
I20260811 01:49:40.156538 12244 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 01:49:40.160429 12261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:40.161990 12244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 01:49:40.162179 12244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "1e6fa03ce9e94e63a2f4a2be663033b7"
format_stamp: "Formatted at 2026-08-11 01:49:40 on dist-test-slave-9gcw"
server_key: "ebdde48e4d3f68fd74fa0988ac75fbb3"
server_key_iv: "554d9ff1dcc01c86f886e64d521ba927"
server_key_version: "encryptionkey@0"
I20260811 01:49:40.162406 12244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:40.179545 12244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:40.184475 12244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:40.184819 12244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:40.205325 12244 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:35817
I20260811 01:49:40.205350 12321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:35817 every 8 connection(s)
I20260811 01:49:40.206861 12244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:49:40.211118 12322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:40.211871 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 12244
I20260811 01:49:40.212111 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:49:40.212793 22361 external_mini_cluster.cc:1428] Setting key c1f7cea4671542d75ed023a2865fd199
I20260811 01:49:40.227121 12322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7: Bootstrap starting.
Aug 11 01:49:40 dist-test-slave-9gcw krb5kdc[12228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:49:40.231374 12322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:40.232510 12322 log.cc:824] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:40.235764 12322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7: No bootstrap required, opened a new log
I20260811 01:49:40.239543 12326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:40.215103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60294 (local address 127.21.214.126:35817)
0811 01:49:40.216044 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:40.216057 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:40.216103 (+ 46us) server_negotiation.cc:407] Connection header received
0811 01:49:40.218571 (+ 2468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:40.218603 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:40.219108 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:40.219521 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:40.221634 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:40.223258 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:40.225290 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:40.226076 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:40.226257 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:40.231408 (+ 5151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:40.231417 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:40.231456 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:40.231515 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:40.234871 (+ 3356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:40.235666 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:40.235672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:40.235679 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:40.235914 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:40.236343 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:40.236354 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:40.236356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:40.236933 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:40.237318 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:40.237817 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 01:49:40.238480 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":449,"threads_started":1}
I20260811 01:49:40.243867 12322 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e6fa03ce9e94e63a2f4a2be663033b7" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 35817 } }
I20260811 01:49:40.244268 12322 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:40.244400 12322 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6fa03ce9e94e63a2f4a2be663033b7, State: Initialized, Role: FOLLOWER
I20260811 01:49:40.244967 12322 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e6fa03ce9e94e63a2f4a2be663033b7" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 35817 } }
I20260811 01:49:40.245234 12322 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:49:40.245406 12322 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:49:40.245581 12322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:40.247895 12322 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e6fa03ce9e94e63a2f4a2be663033b7" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 35817 } }
I20260811 01:49:40.248430 12322 leader_election.cc:302] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e6fa03ce9e94e63a2f4a2be663033b7; no voters:
I20260811 01:49:40.249222 12322 leader_election.cc:288] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:49:40.249338 12328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:49:40.250025 12328 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7 [term 1 LEADER]: Becoming Leader. State: Replica: 1e6fa03ce9e94e63a2f4a2be663033b7, State: Running, Role: LEADER
I20260811 01:49:40.250659 12328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e6fa03ce9e94e63a2f4a2be663033b7" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 35817 } }
I20260811 01:49:40.251577 12322 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:49:40.254004 12329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e6fa03ce9e94e63a2f4a2be663033b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e6fa03ce9e94e63a2f4a2be663033b7" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 35817 } } }
I20260811 01:49:40.253974 12330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e6fa03ce9e94e63a2f4a2be663033b7. Latest consensus state: current_term: 1 leader_uuid: "1e6fa03ce9e94e63a2f4a2be663033b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e6fa03ce9e94e63a2f4a2be663033b7" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 35817 } } }
I20260811 01:49:40.254611 12329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7 [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:40.254679 12330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7 [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:40.261111 12336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:49:40.267418 12336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:49:40.278249 12336 catalog_manager.cc:1347] Generated new cluster ID: 3a54a68470c44f218d28554c37f08c79
I20260811 01:49:40.278400 12336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:49:40.308616 12336 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:49:40.310689 12336 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:49:40.319415 12336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7: Generated new TSK 0
I20260811 01:49:40.320546 12336 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.214.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
Entry for principal kudu/127.21.214.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab.
I20260811 01:49:40.397425 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:35817
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:37421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:49:40.766403 12352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:40.766762 12352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:40.766940 12352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:40.774770 12352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:40.774937 12352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:40.775115 12352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:49:40.785421 12352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:37421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.65.keytab
--principal=kudu/127.21.214.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:35817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:40.786947 12352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:40.788390 12352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:40.802983 12358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:40.802940 12359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:40.803172 12361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:40.804361 12352 server_base.cc:1034] running on GCE node
I20260811 01:49:40.805037 12352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:40.806121 12352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:40.807386 12352 hybrid_clock.cc:648] HybridClock initialized: now 1786412980807339 us; error 45 us; skew 500 ppm
I20260811 01:49:40.807767 12352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:40 dist-test-slave-9gcw krb5kdc[12228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412980, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:40.811383 12352 init.cc:377] Logged in from keytab as kudu/127.21.214.65@KRBTEST.COM (short username kudu)
I20260811 01:49:40.812984 12352 webserver.cc:460] Webserver started at http://127.21.214.65:44751/ using document root <none> and password file <none>
I20260811 01:49:40.813743 12352 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:40.813853 12352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:40.814133 12352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:40.817126 12352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "29fdd2b033d74452863db1e7866eab84"
format_stamp: "Formatted at 2026-08-11 01:49:40 on dist-test-slave-9gcw"
server_key: "474f35aec5cd52e752c344d50214f207"
server_key_iv: "8f7782ee4046f37090eb096a29ef128f"
server_key_version: "encryptionkey@0"
I20260811 01:49:40.817952 12352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "29fdd2b033d74452863db1e7866eab84"
format_stamp: "Formatted at 2026-08-11 01:49:40 on dist-test-slave-9gcw"
server_key: "474f35aec5cd52e752c344d50214f207"
server_key_iv: "8f7782ee4046f37090eb096a29ef128f"
server_key_version: "encryptionkey@0"
I20260811 01:49:40.823798 12352 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 01:49:40.827787 12370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:40.829464 12352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 01:49:40.829684 12352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "29fdd2b033d74452863db1e7866eab84"
format_stamp: "Formatted at 2026-08-11 01:49:40 on dist-test-slave-9gcw"
server_key: "474f35aec5cd52e752c344d50214f207"
server_key_iv: "8f7782ee4046f37090eb096a29ef128f"
server_key_version: "encryptionkey@0"
I20260811 01:49:40.829916 12352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:40.845975 12352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:40.851112 12352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:40.851521 12352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:40.852784 12352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:40.854450 12352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:40.854571 12352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:40.854748 12352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:40.854835 12352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:40.892747 12352 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:38491
I20260811 01:49:40.892791 12503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:38491 every 8 connection(s)
I20260811 01:49:40.894317 12352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:49:40.903510 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 12352
I20260811 01:49:40.903729 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:49:40.904210 22361 external_mini_cluster.cc:1428] Setting key 6d651f84efe778cd78e96eff283ed82d
Aug 11 01:49:40 dist-test-slave-9gcw krb5kdc[12228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412980, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.65@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:49:40.917546 12506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:40.897380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:43015 (local address 127.21.214.126:35817)
0811 01:49:40.897952 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:40.897957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:40.899036 (+ 1079us) server_negotiation.cc:407] Connection header received
0811 01:49:40.901109 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:40.901115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:40.901253 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:40.901393 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:40.903105 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:40.904036 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:40.906228 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:40.906691 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:40.906848 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:40.911017 (+ 4169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:40.911023 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:40.911027 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:40.911061 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:40.913988 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:40.914840 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:40.914846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:40.914848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:40.914977 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:40.915542 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:40.915546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:40.915548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:40.915844 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:40.916020 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:40.916836 (+ 816us) server_negotiation.cc:299] Negotiation successful
0811 01:49:40.917095 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":310,"threads_started":1}
I20260811 01:49:40.918615 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:40.898021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:35817 (local address 127.21.214.65:43015)
0811 01:49:40.898861 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:40.898904 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:40.900802 (+ 1898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:40.901667 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:40.901677 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:40.902272 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:49:40.902910 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:40.902929 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:40.904230 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:40.904235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:40.906040 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:40.906057 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:40.906955 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:40.906967 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:40.907252 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:40.908057 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:49:40.908082 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:49:40.910761 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:49:40.914178 (+ 3417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:49:40.914190 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:49:40.914217 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:49:40.914684 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:49:40.915106 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:49:40.915112 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:49:40.915116 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:49:40.915391 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:49:40.915993 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:49:40.916002 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:49:40.916391 (+ 389us) client_negotiation.cc:769] Sending connection context
0811 01:49:40.916683 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 01:49:40.917344 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":292,"threads_started":1}
I20260811 01:49:40.920511 12504 heartbeater.cc:347] Connected to a master server at 127.21.214.126:35817
I20260811 01:49:40.920886 12504 heartbeater.cc:464] Registering TS with master...
I20260811 01:49:40.921840 12504 heartbeater.cc:511] Master 127.21.214.126:35817 requested a full tablet report, sending...
I20260811 01:49:40.924539 12281 ts_manager.cc:194] Registered new tserver with Master: 29fdd2b033d74452863db1e7866eab84 (127.21.214.65:38491)
I20260811 01:49:40.926883 12281 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.65@KRBTEST.COM'} at 127.21.214.65:43015
WARNING: no policy specified for kudu/127.21.214.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
Entry for principal kudu/127.21.214.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab.
I20260811 01:49:40.981805 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:35817
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:37421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:49:41.355254 12510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:41.355625 12510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:41.355744 12510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:41.363610 12510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:41.363754 12510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:41.363885 12510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:49:41.374694 12510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:37421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.66.keytab
--principal=kudu/127.21.214.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:35817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:41.376164 12510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:41.377660 12510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:41.392652 12516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:41.392786 12519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:41.392690 12517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:41.395437 12510 server_base.cc:1034] running on GCE node
I20260811 01:49:41.396188 12510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:41.397416 12510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:41.398667 12510 hybrid_clock.cc:648] HybridClock initialized: now 1786412981398594 us; error 76 us; skew 500 ppm
I20260811 01:49:41.399087 12510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:41 dist-test-slave-9gcw krb5kdc[12228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412981, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:41.402777 12510 init.cc:377] Logged in from keytab as kudu/127.21.214.66@KRBTEST.COM (short username kudu)
I20260811 01:49:41.404532 12510 webserver.cc:460] Webserver started at http://127.21.214.66:39063/ using document root <none> and password file <none>
I20260811 01:49:41.405320 12510 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:41.405462 12510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:41.405786 12510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:41.409004 12510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "0b454845b3f94d9683c68476279e58e8"
format_stamp: "Formatted at 2026-08-11 01:49:41 on dist-test-slave-9gcw"
server_key: "7822c82ee37b1e03671e6e48073a9665"
server_key_iv: "0481c10156b8986b8343a47bb04d0672"
server_key_version: "encryptionkey@0"
I20260811 01:49:41.409888 12510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "0b454845b3f94d9683c68476279e58e8"
format_stamp: "Formatted at 2026-08-11 01:49:41 on dist-test-slave-9gcw"
server_key: "7822c82ee37b1e03671e6e48073a9665"
server_key_iv: "0481c10156b8986b8343a47bb04d0672"
server_key_version: "encryptionkey@0"
I20260811 01:49:41.415653 12510 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 01:49:41.419838 12526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:41.421483 12510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:49:41.421674 12510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "0b454845b3f94d9683c68476279e58e8"
format_stamp: "Formatted at 2026-08-11 01:49:41 on dist-test-slave-9gcw"
server_key: "7822c82ee37b1e03671e6e48073a9665"
server_key_iv: "0481c10156b8986b8343a47bb04d0672"
server_key_version: "encryptionkey@0"
I20260811 01:49:41.421919 12510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:41.455397 12510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:41.460536 12510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:41.460948 12510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:41.462182 12510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:41.463621 12510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:41.463702 12510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:41.463858 12510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:41.463920 12510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:41.502409 12510 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:42039
I20260811 01:49:41.502445 12658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:42039 every 8 connection(s)
I20260811 01:49:41.504122 12510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:49:41.508441 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 12510
I20260811 01:49:41.508656 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:49:41.509275 22361 external_mini_cluster.cc:1428] Setting key 5208e204c95134294d3444622d10bc4f
Aug 11 01:49:41 dist-test-slave-9gcw krb5kdc[12228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412981, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.66@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:49:41.529821 12661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:41.506966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:36663 (local address 127.21.214.126:35817)
0811 01:49:41.507521 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:41.507527 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:41.508751 (+ 1224us) server_negotiation.cc:407] Connection header received
0811 01:49:41.511063 (+ 2312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:41.511068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:41.511137 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:41.511287 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:41.513059 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:41.514325 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:41.517011 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:41.517796 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:41.517951 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:41.522361 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:41.522370 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:41.522379 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:41.522430 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:41.525982 (+ 3552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:41.527156 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:41.527165 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:41.527172 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:41.527369 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:41.527950 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:41.527953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:41.527956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:41.528250 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:41.528361 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:41.529355 (+ 994us) server_negotiation.cc:299] Negotiation successful
0811 01:49:41.529532 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":322,"threads_started":1}
I20260811 01:49:41.531589 12662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:41.507453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:35817 (local address 127.21.214.66:36663)
0811 01:49:41.508397 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:41.508466 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:41.510746 (+ 2280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:41.511626 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:41.511641 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:41.512240 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:49:41.512853 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:41.512873 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:41.514553 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:41.514558 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:41.516826 (+ 2268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:41.516844 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:41.518073 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:41.518079 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:41.518341 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:41.519281 (+ 940us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:49:41.519311 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:49:41.522082 (+ 2771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:49:41.526134 (+ 4052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:49:41.526146 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:49:41.526170 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:49:41.526954 (+ 784us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:49:41.527522 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:49:41.527529 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:49:41.527536 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:49:41.527811 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:49:41.528396 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:49:41.528408 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:49:41.528913 (+ 505us) client_negotiation.cc:769] Sending connection context
0811 01:49:41.529287 (+ 374us) client_negotiation.cc:240] Negotiation successful
0811 01:49:41.529948 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":372,"threads_started":1}
I20260811 01:49:41.533320 12659 heartbeater.cc:347] Connected to a master server at 127.21.214.126:35817
I20260811 01:49:41.533691 12659 heartbeater.cc:464] Registering TS with master...
I20260811 01:49:41.534487 12659 heartbeater.cc:511] Master 127.21.214.126:35817 requested a full tablet report, sending...
I20260811 01:49:41.536379 12281 ts_manager.cc:194] Registered new tserver with Master: 0b454845b3f94d9683c68476279e58e8 (127.21.214.66:42039)
I20260811 01:49:41.537737 12281 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.66@KRBTEST.COM'} at 127.21.214.66:36663
WARNING: no policy specified for kudu/127.21.214.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
Entry for principal kudu/127.21.214.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab.
I20260811 01:49:41.590555 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:35817
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:37421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:49:41.931084 12504 heartbeater.cc:503] Master 127.21.214.126:35817 was elected leader, sending a full tablet report...
W20260811 01:49:41.956737 12666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:41.957109 12666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:41.957319 12666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:41.964954 12666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:41.965130 12666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:41.965348 12666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:49:41.975647 12666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:37421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.67.keytab
--principal=kudu/127.21.214.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:35817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:41.977116 12666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:41.978569 12666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:41.994900 12671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:41.994843 12672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:41.994902 12674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:41.995342 12666 server_base.cc:1034] running on GCE node
I20260811 01:49:41.996302 12666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:41.997460 12666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:41.998765 12666 hybrid_clock.cc:648] HybridClock initialized: now 1786412981998627 us; error 41 us; skew 500 ppm
I20260811 01:49:41.999125 12666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:42 dist-test-slave-9gcw krb5kdc[12228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412982, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:42.003181 12666 init.cc:377] Logged in from keytab as kudu/127.21.214.67@KRBTEST.COM (short username kudu)
I20260811 01:49:42.004951 12666 webserver.cc:460] Webserver started at http://127.21.214.67:42379/ using document root <none> and password file <none>
I20260811 01:49:42.005837 12666 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:42.006018 12666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:42.006405 12666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:42.009996 12666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "42fdd8c06db446108a992f014c32baaf"
format_stamp: "Formatted at 2026-08-11 01:49:42 on dist-test-slave-9gcw"
server_key: "14a0795751dea558a6ccad875b73496e"
server_key_iv: "8853af1210debbebcf839af0863d54c0"
server_key_version: "encryptionkey@0"
I20260811 01:49:42.010979 12666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "42fdd8c06db446108a992f014c32baaf"
format_stamp: "Formatted at 2026-08-11 01:49:42 on dist-test-slave-9gcw"
server_key: "14a0795751dea558a6ccad875b73496e"
server_key_iv: "8853af1210debbebcf839af0863d54c0"
server_key_version: "encryptionkey@0"
I20260811 01:49:42.017282 12666 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 01:49:42.021049 12683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:42.022783 12666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:49:42.022985 12666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "42fdd8c06db446108a992f014c32baaf"
format_stamp: "Formatted at 2026-08-11 01:49:42 on dist-test-slave-9gcw"
server_key: "14a0795751dea558a6ccad875b73496e"
server_key_iv: "8853af1210debbebcf839af0863d54c0"
server_key_version: "encryptionkey@0"
I20260811 01:49:42.023232 12666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:42.053217 12666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:42.058707 12666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:42.059069 12666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:42.060209 12666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:42.061916 12666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:42.061997 12666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:42.062189 12666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:42.062249 12666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:42.100610 12666 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:43517
I20260811 01:49:42.100682 12824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:43517 every 8 connection(s)
I20260811 01:49:42.102268 12666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:49:42.105736 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 12666
I20260811 01:49:42.105955 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:49:42.106652 22361 external_mini_cluster.cc:1428] Setting key 3e8a537d7bf48f728ce687ad71596344
Aug 11 01:49:42 dist-test-slave-9gcw krb5kdc[12228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412982, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.67@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:49:42.125931 12827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:42.105248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:47793 (local address 127.21.214.126:35817)
0811 01:49:42.105761 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:42.105765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:42.107238 (+ 1473us) server_negotiation.cc:407] Connection header received
0811 01:49:42.109843 (+ 2605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:42.109852 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:42.109981 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:42.110073 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:42.111755 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.112596 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.114372 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.114775 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.114875 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:42.118445 (+ 3570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:42.118450 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:42.118453 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:42.118497 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:42.121690 (+ 3193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:42.122661 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:42.122666 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:42.122703 (+ 37us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:42.122957 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:42.123667 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:42.123675 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:42.123677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:42.124020 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:42.124250 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:42.125115 (+ 865us) server_negotiation.cc:299] Negotiation successful
0811 01:49:42.125430 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":261,"threads_started":1}
I20260811 01:49:42.127265 12828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:42.106073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:35817 (local address 127.21.214.67:47793)
0811 01:49:42.107016 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:42.107055 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:42.109472 (+ 2417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:42.110380 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:42.110389 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:42.110962 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:49:42.111574 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:42.111592 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.112810 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.112813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:42.114227 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:42.114245 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.114974 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.114978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:42.115179 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:42.115850 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:49:42.115872 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:49:42.118228 (+ 2356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:49:42.121849 (+ 3621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:49:42.121859 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:49:42.121879 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:49:42.122463 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:49:42.123132 (+ 669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:49:42.123138 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:49:42.123145 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:49:42.123421 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:49:42.124195 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:49:42.124206 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:49:42.124634 (+ 428us) client_negotiation.cc:769] Sending connection context
0811 01:49:42.124961 (+ 327us) client_negotiation.cc:240] Negotiation successful
0811 01:49:42.125651 (+ 690us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":328,"threads_started":1}
I20260811 01:49:42.129109 12825 heartbeater.cc:347] Connected to a master server at 127.21.214.126:35817
I20260811 01:49:42.129748 12825 heartbeater.cc:464] Registering TS with master...
I20260811 01:49:42.130548 12825 heartbeater.cc:511] Master 127.21.214.126:35817 requested a full tablet report, sending...
I20260811 01:49:42.132232 12281 ts_manager.cc:194] Registered new tserver with Master: 42fdd8c06db446108a992f014c32baaf (127.21.214.67:43517)
I20260811 01:49:42.133093 12281 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.214.67@KRBTEST.COM'} at 127.21.214.67:47793
I20260811 01:49:42.135924 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:49:42.154819 12827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:42.140142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60304 (local address 127.21.214.126:35817)
0811 01:49:42.140356 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:42.140362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:42.140913 (+ 551us) server_negotiation.cc:407] Connection header received
0811 01:49:42.141032 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:42.141037 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:42.141138 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:42.141315 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:42.142878 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.143705 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.145079 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.145640 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.145762 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:42.148359 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:42.148365 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:42.148369 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:42.148415 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:42.151724 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:42.152505 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:42.152511 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:42.152513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:42.152779 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:42.153211 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:42.153216 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:42.153218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:42.153686 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:42.153902 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:42.154249 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 01:49:42.154454 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 01:49:42.159231 12281 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:49:42.166613 12281 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:49:42.191874 12839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:42.179935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:43517 (local address 127.0.0.1:36002)
0811 01:49:42.180826 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:42.180879 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:42.181066 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:42.183578 (+ 2512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:42.183585 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:42.183617 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:42.184012 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:42.184031 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.185965 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.185973 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:42.188482 (+ 2509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:42.188498 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.190762 (+ 2264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.190767 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:42.190977 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:42.191033 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 01:49:42.191182 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 01:49:42.191396 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":4,"thread_start_us":406,"threads_started":1}
I20260811 01:49:42.191893 12840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:42.180210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36002 (local address 127.21.214.67:43517)
0811 01:49:42.181908 (+ 1698us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:42.181920 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:42.181962 (+ 42us) server_negotiation.cc:407] Connection header received
0811 01:49:42.182070 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:42.182079 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:42.182380 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:42.182598 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:42.184260 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.185718 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.188670 (+ 2952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.190270 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.190441 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:42.190635 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:42.191318 (+ 683us) server_negotiation.cc:299] Negotiation successful
0811 01:49:42.191511 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1509,"thread_start_us":1342,"threads_started":1}
I20260811 01:49:42.193432 12844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:42.182306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53662 (local address 127.21.214.65:38491)
0811 01:49:42.183500 (+ 1194us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:42.183510 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:42.183539 (+ 29us) server_negotiation.cc:407] Connection header received
0811 01:49:42.183631 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:42.183638 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:42.183911 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:42.184850 (+ 939us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:42.185128 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.186768 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.189536 (+ 2768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.190756 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.190940 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:42.191173 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:42.192847 (+ 1674us) server_negotiation.cc:299] Negotiation successful
0811 01:49:42.193068 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1016,"thread_start_us":853,"threads_started":1}
I20260811 01:49:42.193480 12841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:42.181641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:38491 (local address 127.0.0.1:53662)
0811 01:49:42.182780 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:42.182805 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:42.182915 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:42.184148 (+ 1233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:42.184154 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:42.184186 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:42.184613 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:42.184625 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.186918 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.186923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:42.189355 (+ 2432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:42.189368 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.192387 (+ 3019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.192399 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:42.192611 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:42.192657 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 01:49:42.192793 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 01:49:42.192930 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":992,"mutex_wait_us":54,"thread_start_us":924,"threads_started":1}
I20260811 01:49:42.194402 12843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:42.181693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60142 (local address 127.21.214.66:42039)
0811 01:49:42.184760 (+ 3067us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:42.184770 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:42.184802 (+ 32us) server_negotiation.cc:407] Connection header received
0811 01:49:42.184908 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:42.184918 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:42.185184 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:42.186018 (+ 834us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:42.186254 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.187954 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.190440 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.192301 (+ 1861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.193608 (+ 1307us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:42.193820 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:42.193906 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 01:49:42.194070 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2928,"thread_start_us":2812,"threads_started":1}
I20260811 01:49:42.194749 12842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:42.181236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:42039 (local address 127.0.0.1:60142)
0811 01:49:42.182147 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:42.182169 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:42.182282 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:42.185424 (+ 3142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:42.185428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:42.185456 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:42.185818 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:42.185830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.188083 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.188089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:42.190237 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:42.190254 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.192595 (+ 2341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.192601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:42.192797 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:42.192832 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 01:49:42.192901 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 01:49:42.193030 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"mutex_wait_us":1,"thread_start_us":1517,"threads_started":1}
I20260811 01:49:42.196579 12740 tablet_service.cc:1467] Processing CreateTablet for tablet 4859502110f148459ec4b339822aa763 (DEFAULT_TABLE table=test-table [id=6dcec3d6ee904647ae01568e3a62ceb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:49:42.196945 12428 tablet_service.cc:1467] Processing CreateTablet for tablet 4859502110f148459ec4b339822aa763 (DEFAULT_TABLE table=test-table [id=6dcec3d6ee904647ae01568e3a62ceb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:49:42.198319 12740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4859502110f148459ec4b339822aa763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:42.198370 12428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4859502110f148459ec4b339822aa763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:42.198465 12587 tablet_service.cc:1467] Processing CreateTablet for tablet 4859502110f148459ec4b339822aa763 (DEFAULT_TABLE table=test-table [id=6dcec3d6ee904647ae01568e3a62ceb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:49:42.199831 12587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4859502110f148459ec4b339822aa763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:42.216740 12847 tablet_bootstrap.cc:492] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf: Bootstrap starting.
I20260811 01:49:42.224716 12845 tablet_bootstrap.cc:492] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84: Bootstrap starting.
I20260811 01:49:42.227946 12848 tablet_bootstrap.cc:492] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8: Bootstrap starting.
I20260811 01:49:42.230180 12847 tablet_bootstrap.cc:654] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:42.231467 12848 tablet_bootstrap.cc:654] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:42.232612 12848 log.cc:824] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:42.232789 12847 log.cc:824] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:42.233470 12845 tablet_bootstrap.cc:654] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:42.234973 12845 log.cc:824] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:42.236225 12848 tablet_bootstrap.cc:492] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8: No bootstrap required, opened a new log
I20260811 01:49:42.236548 12848 ts_tablet_manager.cc:1397] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260811 01:49:42.237360 12847 tablet_bootstrap.cc:492] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf: No bootstrap required, opened a new log
I20260811 01:49:42.237629 12847 ts_tablet_manager.cc:1397] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.004s
I20260811 01:49:42.238391 12845 tablet_bootstrap.cc:492] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84: No bootstrap required, opened a new log
I20260811 01:49:42.238723 12845 ts_tablet_manager.cc:1397] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.002s
I20260811 01:49:42.245800 12847 raft_consensus.cc:348] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fdd8c06db446108a992f014c32baaf" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 43517 } } peers { permanent_uuid: "0b454845b3f94d9683c68476279e58e8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42039 } } peers { permanent_uuid: "29fdd2b033d74452863db1e7866eab84" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38491 } }
I20260811 01:49:42.246227 12847 raft_consensus.cc:374] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:42.246335 12847 raft_consensus.cc:729] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42fdd8c06db446108a992f014c32baaf, State: Initialized, Role: FOLLOWER
I20260811 01:49:42.246973 12847 consensus_queue.cc:260] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fdd8c06db446108a992f014c32baaf" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 43517 } } peers { permanent_uuid: "0b454845b3f94d9683c68476279e58e8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42039 } } peers { permanent_uuid: "29fdd2b033d74452863db1e7866eab84" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38491 } }
I20260811 01:49:42.247813 12848 raft_consensus.cc:348] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fdd8c06db446108a992f014c32baaf" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 43517 } } peers { permanent_uuid: "0b454845b3f94d9683c68476279e58e8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42039 } } peers { permanent_uuid: "29fdd2b033d74452863db1e7866eab84" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38491 } }
I20260811 01:49:42.248124 12825 heartbeater.cc:503] Master 127.21.214.126:35817 was elected leader, sending a full tablet report...
I20260811 01:49:42.248282 12848 raft_consensus.cc:374] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:42.248390 12848 raft_consensus.cc:729] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b454845b3f94d9683c68476279e58e8, State: Initialized, Role: FOLLOWER
I20260811 01:49:42.249020 12848 consensus_queue.cc:260] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fdd8c06db446108a992f014c32baaf" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 43517 } } peers { permanent_uuid: "0b454845b3f94d9683c68476279e58e8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42039 } } peers { permanent_uuid: "29fdd2b033d74452863db1e7866eab84" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38491 } }
I20260811 01:49:42.249750 12845 raft_consensus.cc:348] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fdd8c06db446108a992f014c32baaf" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 43517 } } peers { permanent_uuid: "0b454845b3f94d9683c68476279e58e8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42039 } } peers { permanent_uuid: "29fdd2b033d74452863db1e7866eab84" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38491 } }
I20260811 01:49:42.250332 12845 raft_consensus.cc:374] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:42.250382 12847 ts_tablet_manager.cc:1428] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260811 01:49:42.250455 12845 raft_consensus.cc:729] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29fdd2b033d74452863db1e7866eab84, State: Initialized, Role: FOLLOWER
I20260811 01:49:42.251189 12845 consensus_queue.cc:260] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fdd8c06db446108a992f014c32baaf" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 43517 } } peers { permanent_uuid: "0b454845b3f94d9683c68476279e58e8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42039 } } peers { permanent_uuid: "29fdd2b033d74452863db1e7866eab84" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38491 } }
I20260811 01:49:42.253569 12845 ts_tablet_manager.cc:1428] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84: Time spent starting tablet: real 0.015s user 0.009s sys 0.006s
I20260811 01:49:42.256217 12659 heartbeater.cc:503] Master 127.21.214.126:35817 was elected leader, sending a full tablet report...
I20260811 01:49:42.257345 12848 ts_tablet_manager.cc:1428] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8: Time spent starting tablet: real 0.021s user 0.019s sys 0.002s
W20260811 01:49:42.354200 12826 tablet.cc:2369] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:49:42.397496 12505 tablet.cc:2369] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:49:42.404148 12857 raft_consensus.cc:482] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:49:42.404500 12857 raft_consensus.cc:504] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fdd8c06db446108a992f014c32baaf" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 43517 } } peers { permanent_uuid: "0b454845b3f94d9683c68476279e58e8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42039 } } peers { permanent_uuid: "29fdd2b033d74452863db1e7866eab84" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38491 } }
I20260811 01:49:42.406471 12857 leader_election.cc:288] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42fdd8c06db446108a992f014c32baaf (127.21.214.67:43517), 0b454845b3f94d9683c68476279e58e8 (127.21.214.66:42039)
I20260811 01:49:42.414021 12840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:42.406949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:40107 (local address 127.21.214.67:43517)
0811 01:49:42.407220 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:42.407228 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:42.407797 (+ 569us) server_negotiation.cc:407] Connection header received
0811 01:49:42.408001 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:42.408009 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:42.408109 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:42.408209 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:42.408949 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.409938 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.411910 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.412887 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.413010 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:42.413063 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:42.413547 (+ 484us) server_negotiation.cc:299] Negotiation successful
0811 01:49:42.413749 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 01:49:42.414031 12858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:42.406925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:43517 (local address 127.21.214.65:40107)
0811 01:49:42.407607 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:42.407623 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:42.407829 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:42.408340 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:42.408351 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:42.408375 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:42.408794 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:42.408807 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.410070 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.410076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:42.411741 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:42.411754 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.413106 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.413113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:42.413331 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:42.413373 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 01:49:42.413488 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 01:49:42.413635 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":1,"thread_start_us":390,"threads_started":1}
I20260811 01:49:42.414460 12843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:42.408044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:43761 (local address 127.21.214.66:42039)
0811 01:49:42.408293 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:42.408301 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:42.408567 (+ 266us) server_negotiation.cc:407] Connection header received
0811 01:49:42.408704 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:42.408712 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:42.408791 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:42.408902 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:42.409556 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.410450 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.412382 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.413437 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.413582 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:42.413649 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:42.414041 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 01:49:42.414192 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 01:49:42.414958 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:42.407796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:42039 (local address 127.21.214.65:43761)
0811 01:49:42.408458 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:42.408474 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:42.408580 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:42.408935 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:42.408939 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:42.408963 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:42.409400 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:42.409414 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.410600 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.410604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:42.412221 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:42.412237 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.413667 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.413674 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:42.413842 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:42.413882 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 01:49:42.413990 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 01:49:42.414706 (+ 716us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":300,"threads_started":1}
I20260811 01:49:42.415068 12765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4859502110f148459ec4b339822aa763" candidate_uuid: "29fdd2b033d74452863db1e7866eab84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42fdd8c06db446108a992f014c32baaf" is_pre_election: true
I20260811 01:49:42.415719 12765 raft_consensus.cc:2454] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29fdd2b033d74452863db1e7866eab84 in term 0.
I20260811 01:49:42.416047 12608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4859502110f148459ec4b339822aa763" candidate_uuid: "29fdd2b033d74452863db1e7866eab84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b454845b3f94d9683c68476279e58e8" is_pre_election: true
I20260811 01:49:42.416447 12372 leader_election.cc:302] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29fdd2b033d74452863db1e7866eab84, 42fdd8c06db446108a992f014c32baaf; no voters:
I20260811 01:49:42.416633 12608 raft_consensus.cc:2454] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29fdd2b033d74452863db1e7866eab84 in term 0.
I20260811 01:49:42.416867 12857 raft_consensus.cc:2781] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:49:42.417016 12857 raft_consensus.cc:482] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:49:42.417109 12857 raft_consensus.cc:3037] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:42.419684 12857 raft_consensus.cc:504] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fdd8c06db446108a992f014c32baaf" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 43517 } } peers { permanent_uuid: "0b454845b3f94d9683c68476279e58e8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42039 } } peers { permanent_uuid: "29fdd2b033d74452863db1e7866eab84" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38491 } }
I20260811 01:49:42.420404 12857 leader_election.cc:288] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [CANDIDATE]: Term 1 election: Requested vote from peers 42fdd8c06db446108a992f014c32baaf (127.21.214.67:43517), 0b454845b3f94d9683c68476279e58e8 (127.21.214.66:42039)
I20260811 01:49:42.420995 12765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4859502110f148459ec4b339822aa763" candidate_uuid: "29fdd2b033d74452863db1e7866eab84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42fdd8c06db446108a992f014c32baaf"
I20260811 01:49:42.421003 12608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4859502110f148459ec4b339822aa763" candidate_uuid: "29fdd2b033d74452863db1e7866eab84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b454845b3f94d9683c68476279e58e8"
I20260811 01:49:42.421240 12765 raft_consensus.cc:3037] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:42.421305 12608 raft_consensus.cc:3037] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:42.423605 12765 raft_consensus.cc:2454] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29fdd2b033d74452863db1e7866eab84 in term 1.
I20260811 01:49:42.424077 12372 leader_election.cc:302] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29fdd2b033d74452863db1e7866eab84, 42fdd8c06db446108a992f014c32baaf; no voters:
I20260811 01:49:42.424247 12608 raft_consensus.cc:2454] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29fdd2b033d74452863db1e7866eab84 in term 1.
I20260811 01:49:42.424417 12857 raft_consensus.cc:2781] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:49:42.424991 12857 raft_consensus.cc:686] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [term 1 LEADER]: Becoming Leader. State: Replica: 29fdd2b033d74452863db1e7866eab84, State: Running, Role: LEADER
I20260811 01:49:42.425733 12857 consensus_queue.cc:237] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fdd8c06db446108a992f014c32baaf" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 43517 } } peers { permanent_uuid: "0b454845b3f94d9683c68476279e58e8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42039 } } peers { permanent_uuid: "29fdd2b033d74452863db1e7866eab84" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38491 } }
I20260811 01:49:42.431643 12280 catalog_manager.cc:5495] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29fdd2b033d74452863db1e7866eab84 (127.21.214.65). New cstate: current_term: 1 leader_uuid: "29fdd2b033d74452863db1e7866eab84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42fdd8c06db446108a992f014c32baaf" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 43517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b454845b3f94d9683c68476279e58e8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29fdd2b033d74452863db1e7866eab84" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 38491 } health_report { overall_health: HEALTHY } } }
I20260811 01:49:42.490748 12844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:42.484787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33294 (local address 127.21.214.65:38491)
0811 01:49:42.485060 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:42.485067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:42.485200 (+ 133us) server_negotiation.cc:407] Connection header received
0811 01:49:42.485493 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:42.485498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:42.485599 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:42.485778 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:49:42.486247 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.486971 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.488402 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.488831 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.488975 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:42.489573 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:49:42.490132 (+ 559us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:49:42.490317 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:42.490376 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 01:49:42.490463 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260811 01:49:42.499949 12608 raft_consensus.cc:1261] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8 [term 1 FOLLOWER]: Refusing update from remote peer 29fdd2b033d74452863db1e7866eab84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:49:42.499948 12765 raft_consensus.cc:1261] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf [term 1 FOLLOWER]: Refusing update from remote peer 29fdd2b033d74452863db1e7866eab84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:49:42.501101 12860 consensus_queue.cc:1035] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b454845b3f94d9683c68476279e58e8" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 42039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:49:42.501627 12857 consensus_queue.cc:1035] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42fdd8c06db446108a992f014c32baaf" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 43517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 01:49:42.506543 12660 tablet.cc:2369] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:49:42.530375 12866 mvcc.cc:205] Tried to move back new op lower bound from 7317147576307806208 to 7317147576020779008. Current Snapshot: MvccSnapshot[applied={T|T < 7317147576307806208}]
I20260811 01:49:42.531844 12281 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:49:42.532155 12281 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:49:42.537344 12281 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e6fa03ce9e94e63a2f4a2be663033b7: Sending DeleteTablet for 3 replicas of tablet 4859502110f148459ec4b339822aa763
I20260811 01:49:42.538758 12428 tablet_service.cc:1514] Processing DeleteTablet for tablet 4859502110f148459ec4b339822aa763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:42 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:53662
I20260811 01:49:42.538839 12427 tablet_service.cc:1514] Processing DeleteTablet for tablet 4859502110f148459ec4b339822aa763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:42 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:53662
I20260811 01:49:42.539023 12587 tablet_service.cc:1514] Processing DeleteTablet for tablet 4859502110f148459ec4b339822aa763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:42 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:60142
I20260811 01:49:42.539583 12586 tablet_service.cc:1514] Processing DeleteTablet for tablet 4859502110f148459ec4b339822aa763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:42 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:60142
I20260811 01:49:42.540014 12740 tablet_service.cc:1514] Processing DeleteTablet for tablet 4859502110f148459ec4b339822aa763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:42 UTC) from {username='kudu', principal='kudu/127.21.214.126@KRBTEST.COM'} at 127.0.0.1:36002
I20260811 01:49:42.541312 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 12352
I20260811 01:49:42.552887 12880 tablet_replica.cc:330] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf: stopping tablet replica
I20260811 01:49:42.553565 12880 raft_consensus.cc:2229] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:49:42.554272 12880 raft_consensus.cc:2258] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:49:42.557044 12880 ts_tablet_manager.cc:1905] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:49:42.566591 12880 ts_tablet_manager.cc:1918] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:49:42.566793 12880 log.cc:1196] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/4859502110f148459ec4b339822aa763
I20260811 01:49:42.567391 12880 ts_tablet_manager.cc:1939] T 4859502110f148459ec4b339822aa763 P 42fdd8c06db446108a992f014c32baaf: Deleting consensus metadata
I20260811 01:49:42.567997 12879 tablet_replica.cc:330] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8: stopping tablet replica
I20260811 01:49:42.568547 12879 raft_consensus.cc:2229] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:49:42.569218 12879 raft_consensus.cc:2258] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:49:42.569703 12265 catalog_manager.cc:4841] TS 42fdd8c06db446108a992f014c32baaf (127.21.214.67:43517): tablet 4859502110f148459ec4b339822aa763 (table test-table [id=6dcec3d6ee904647ae01568e3a62ceb9]) successfully deleted
I20260811 01:49:42.571972 12878 tablet_replica.cc:330] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84: stopping tablet replica
I20260811 01:49:42.572501 12878 raft_consensus.cc:2229] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:49:42.573266 12878 raft_consensus.cc:2258] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:49:42.575551 12879 ts_tablet_manager.cc:1905] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:49:42.579020 12878 ts_tablet_manager.cc:1905] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 01:49:42 dist-test-slave-9gcw krb5kdc[12228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.65@KRBTEST.COM
I20260811 01:49:42.582624 12879 ts_tablet_manager.cc:1918] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:49:42.582815 12879 log.cc:1196] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/4859502110f148459ec4b339822aa763
I20260811 01:49:42.583355 12879 ts_tablet_manager.cc:1939] T 4859502110f148459ec4b339822aa763 P 0b454845b3f94d9683c68476279e58e8: Deleting consensus metadata
I20260811 01:49:42.585532 12265 catalog_manager.cc:4841] TS 0b454845b3f94d9683c68476279e58e8 (127.21.214.66:42039): tablet 4859502110f148459ec4b339822aa763 (table test-table [id=6dcec3d6ee904647ae01568e3a62ceb9]) successfully deleted
W20260811 01:49:42.586683 12265 catalog_manager.cc:4808] TS 0b454845b3f94d9683c68476279e58e8 (127.21.214.66:42039): delete failed for tablet 4859502110f148459ec4b339822aa763 because the tablet was not found. No further retry: Not found: Tablet not found: 4859502110f148459ec4b339822aa763
I20260811 01:49:42.588080 12878 ts_tablet_manager.cc:1918] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:49:42.588285 12878 log.cc:1196] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/4859502110f148459ec4b339822aa763
I20260811 01:49:42.588733 12878 ts_tablet_manager.cc:1939] T 4859502110f148459ec4b339822aa763 P 29fdd2b033d74452863db1e7866eab84: Deleting consensus metadata
I20260811 01:49:42.589893 12844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:42.543048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33304 (local address 127.21.214.65:38491)
0811 01:49:42.543300 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:42.543309 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:42.567194 (+ 23885us) server_negotiation.cc:407] Connection header received
0811 01:49:42.569709 (+ 2515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:42.569717 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:42.569821 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:42.569912 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:42.571446 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.572780 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.574819 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.575560 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.575781 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:42.581414 (+ 5633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:42.581430 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:42.581450 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:42.581504 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:42.585932 (+ 4428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:42.586951 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:42.586962 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:42.586966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:42.587163 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:42.587684 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:42.587690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:42.587694 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:42.588159 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:42.588532 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:42.589229 (+ 697us) server_negotiation.cc:299] Negotiation successful
0811 01:49:42.589489 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 01:49:42.590390 12263 catalog_manager.cc:4841] TS 29fdd2b033d74452863db1e7866eab84 (127.21.214.65:38491): tablet 4859502110f148459ec4b339822aa763 (table test-table [id=6dcec3d6ee904647ae01568e3a62ceb9]) successfully deleted
W20260811 01:49:42.591068 12263 catalog_manager.cc:4808] TS 29fdd2b033d74452863db1e7866eab84 (127.21.214.65:38491): delete failed for tablet 4859502110f148459ec4b339822aa763 because the tablet was not found. No further retry: Not found: Tablet not found: 4859502110f148459ec4b339822aa763
I20260811 01:49:42.591545 12497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:42.692618 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 12352
I20260811 01:49:42.716949 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 12510
Aug 11 01:49:42 dist-test-slave-9gcw krb5kdc[12228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.66@KRBTEST.COM
I20260811 01:49:42.739850 12843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:42.717834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33030 (local address 127.21.214.66:42039)
0811 01:49:42.718053 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:42.718060 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:42.718163 (+ 103us) server_negotiation.cc:407] Connection header received
0811 01:49:42.718406 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:42.718411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:42.718482 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:42.718587 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:42.719928 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.720660 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.721944 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.722332 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.722470 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:42.727736 (+ 5266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:42.727757 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:42.727779 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:42.727827 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:42.736350 (+ 8523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:42.737290 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:42.737300 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:42.737304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:42.737475 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:42.737933 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:42.737938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:42.737941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:42.738448 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:42.738847 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:42.739155 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 01:49:42.739453 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 01:49:42.740850 12653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:42.832683 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 12510
I20260811 01:49:42.855888 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 12666
Aug 11 01:49:42 dist-test-slave-9gcw krb5kdc[12228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.67@KRBTEST.COM
I20260811 01:49:42.872365 12840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:42.856871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50568 (local address 127.21.214.67:43517)
0811 01:49:42.857097 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:42.857103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:42.857124 (+ 21us) server_negotiation.cc:407] Connection header received
0811 01:49:42.857281 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:42.857285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:42.857349 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:42.857441 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:42.858936 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.859799 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.861210 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:42.861693 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:42.861790 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:42.866495 (+ 4705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:42.866510 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:42.866528 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:42.866571 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:42.869731 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:42.870385 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:42.870393 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:42.870395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:42.870557 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:42.870894 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:42.870897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:42.870899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:42.871288 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:42.871501 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:42.871791 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 01:49:42.871961 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 01:49:42.873063 12818 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:42.971773 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 12666
I20260811 01:49:42.997046 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 12244
I20260811 01:49:42.998327 12316 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:43.114575 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 12244
2026-08-11T01:49:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4964 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T01:49:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:49:43Z Disabled control of system clock
I20260811 01:49:43.220160 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:43969
--webserver_interface=127.21.214.126
--webserver_port=0
--builtin_ntp_servers=127.21.214.84:35803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:43969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 01:49:43.611790 12892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:43.612152 12892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:43.612316 12892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:43.620203 12892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:49:43.620368 12892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:43.620455 12892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:43.620570 12892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:49:43.620674 12892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:49:43.631704 12892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:35803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:43969
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:43969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:43.633316 12892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:43.635514 12892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:43.649696 12900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:43.649782 12902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:43.649809 12898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:43.651544 12892 server_base.cc:1034] running on GCE node
I20260811 01:49:43.652482 12892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:43.654209 12892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:43.655514 12892 hybrid_clock.cc:648] HybridClock initialized: now 1786412983655442 us; error 50 us; skew 500 ppm
I20260811 01:49:43.655961 12892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:49:43.659061 12892 webserver.cc:460] Webserver started at http://127.21.214.126:33311/ using document root <none> and password file <none>
I20260811 01:49:43.659885 12892 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:43.660043 12892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:43.660408 12892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:43.663687 12892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "aa55308b13c84e83ae45ff025e078afb"
format_stamp: "Formatted at 2026-08-11 01:49:43 on dist-test-slave-9gcw"
server_key: "26b19645aa34d1910f6a99d643784697"
server_key_iv: "e251e96544bb20d8b9066b81617c3e7c"
server_key_version: "encryptionkey@0"
I20260811 01:49:43.664625 12892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "aa55308b13c84e83ae45ff025e078afb"
format_stamp: "Formatted at 2026-08-11 01:49:43 on dist-test-slave-9gcw"
server_key: "26b19645aa34d1910f6a99d643784697"
server_key_iv: "e251e96544bb20d8b9066b81617c3e7c"
server_key_version: "encryptionkey@0"
I20260811 01:49:43.670439 12892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 01:49:43.674463 12909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:43.676110 12892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260811 01:49:43.676278 12892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "aa55308b13c84e83ae45ff025e078afb"
format_stamp: "Formatted at 2026-08-11 01:49:43 on dist-test-slave-9gcw"
server_key: "26b19645aa34d1910f6a99d643784697"
server_key_iv: "e251e96544bb20d8b9066b81617c3e7c"
server_key_version: "encryptionkey@0"
I20260811 01:49:43.676478 12892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:43.697290 12892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:43.698160 12892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:43.698421 12892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:43.719304 12892 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:43969
I20260811 01:49:43.719326 12975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:43969 every 8 connection(s)
I20260811 01:49:43.720811 12892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:49:43.724951 12977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:43.726007 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 12892
I20260811 01:49:43.726248 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:49:43.726833 22361 external_mini_cluster.cc:1428] Setting key 0c9bbc6f801efbbb2540b3fc69526cbd
I20260811 01:49:43.739259 12977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb: Bootstrap starting.
I20260811 01:49:43.741652 12980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:43.729254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48324 (local address 127.21.214.126:43969)
0811 01:49:43.730402 (+ 1148us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:43.730417 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:43.730461 (+ 44us) server_negotiation.cc:407] Connection header received
0811 01:49:43.732235 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:43.732261 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:43.732759 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:43.733208 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:43.733795 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:43.735214 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:43.736661 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:43.737486 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:43.737700 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:43.739019 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:43.739027 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:43.739044 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:43.739695 (+ 651us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:43.739787 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:43.739817 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:43.739957 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:43.740207 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 01:49:43.740545 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":421,"threads_started":1}
I20260811 01:49:43.743145 12977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:43.744585 12977 log.cc:824] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:43.747617 12977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb: No bootstrap required, opened a new log
I20260811 01:49:43.756155 12977 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa55308b13c84e83ae45ff025e078afb" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43969 } }
I20260811 01:49:43.756582 12977 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:43.756709 12977 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa55308b13c84e83ae45ff025e078afb, State: Initialized, Role: FOLLOWER
I20260811 01:49:43.757416 12977 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa55308b13c84e83ae45ff025e078afb" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43969 } }
I20260811 01:49:43.757666 12977 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:49:43.757826 12977 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:49:43.758003 12977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:43.760324 12977 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa55308b13c84e83ae45ff025e078afb" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43969 } }
I20260811 01:49:43.760882 12977 leader_election.cc:302] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa55308b13c84e83ae45ff025e078afb; no voters:
I20260811 01:49:43.761607 12977 leader_election.cc:288] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:49:43.761811 12982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:49:43.763522 12982 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb [term 1 LEADER]: Becoming Leader. State: Replica: aa55308b13c84e83ae45ff025e078afb, State: Running, Role: LEADER
I20260811 01:49:43.764074 12977 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:49:43.764223 12982 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa55308b13c84e83ae45ff025e078afb" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43969 } }
I20260811 01:49:43.767846 12983 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa55308b13c84e83ae45ff025e078afb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa55308b13c84e83ae45ff025e078afb" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43969 } } }
I20260811 01:49:43.768500 12983 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:43.767863 12985 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa55308b13c84e83ae45ff025e078afb. Latest consensus state: current_term: 1 leader_uuid: "aa55308b13c84e83ae45ff025e078afb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa55308b13c84e83ae45ff025e078afb" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43969 } } }
I20260811 01:49:43.769201 12985 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:43.769896 12994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:49:43.775068 12994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:49:43.785347 12994 catalog_manager.cc:1347] Generated new cluster ID: b0a233c7974d49618179cbf10336a1d5
I20260811 01:49:43.785512 12994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:49:43.798888 12994 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:49:43.800980 12994 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:49:43.811141 12994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb: Generated new TSK 0
I20260811 01:49:43.812403 12994 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:49:43.822467 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.65:0
--local_ip_for_outbound_sockets=127.21.214.65
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:43969
--builtin_ntp_servers=127.21.214.84:35803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 01:49:44.214128 13002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:44.214486 13002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:44.214684 13002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:44.223053 13002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:44.223242 13002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:44.223433 13002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.65
I20260811 01:49:44.233335 13002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:35803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.214.65
--webserver_port=0
--tserver_master_addrs=127.21.214.126:43969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.65
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:44.234782 13002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:44.236775 13002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:44.251588 13009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:44.251538 13010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:44.251566 13012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:44.252521 13002 server_base.cc:1034] running on GCE node
I20260811 01:49:44.253307 13002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:44.254410 13002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:44.255730 13002 hybrid_clock.cc:648] HybridClock initialized: now 1786412984255634 us; error 54 us; skew 500 ppm
I20260811 01:49:44.256162 13002 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:49:44.258991 13002 webserver.cc:460] Webserver started at http://127.21.214.65:33661/ using document root <none> and password file <none>
I20260811 01:49:44.259783 13002 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:44.259929 13002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:44.260337 13002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:44.263604 13002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-0/data/instance:
uuid: "1e70d921f7fb42a98530a0cc4d8d9ea9"
format_stamp: "Formatted at 2026-08-11 01:49:44 on dist-test-slave-9gcw"
server_key: "82673c067b261797bd3fb5139580d89b"
server_key_iv: "f3aaa257642dba31aa51d60caa0b207e"
server_key_version: "encryptionkey@0"
I20260811 01:49:44.264703 13002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance:
uuid: "1e70d921f7fb42a98530a0cc4d8d9ea9"
format_stamp: "Formatted at 2026-08-11 01:49:44 on dist-test-slave-9gcw"
server_key: "82673c067b261797bd3fb5139580d89b"
server_key_iv: "f3aaa257642dba31aa51d60caa0b207e"
server_key_version: "encryptionkey@0"
I20260811 01:49:44.271342 13002 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 01:49:44.275629 13018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:44.277361 13002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:49:44.277583 13002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-0/wal
uuid: "1e70d921f7fb42a98530a0cc4d8d9ea9"
format_stamp: "Formatted at 2026-08-11 01:49:44 on dist-test-slave-9gcw"
server_key: "82673c067b261797bd3fb5139580d89b"
server_key_iv: "f3aaa257642dba31aa51d60caa0b207e"
server_key_version: "encryptionkey@0"
I20260811 01:49:44.277854 13002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:44.296893 13002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:44.297964 13002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:44.298357 13002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:44.299484 13002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:44.301250 13002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:44.301385 13002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:44.301522 13002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:44.301611 13002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:44.337697 13002 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.65:39587
I20260811 01:49:44.337765 13152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.65:39587 every 8 connection(s)
I20260811 01:49:44.339170 13002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-0/data/info.pb
I20260811 01:49:44.339550 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 13002
I20260811 01:49:44.339754 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-0/wal/instance
I20260811 01:49:44.340549 22361 external_mini_cluster.cc:1428] Setting key a84d162c510c3dbd97159f39bfaaf2b1
I20260811 01:49:44.344357 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.66:0
--local_ip_for_outbound_sockets=127.21.214.66
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:43969
--builtin_ntp_servers=127.21.214.84:35803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 01:49:44.356180 13156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:44.342482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:50329 (local address 127.21.214.126:43969)
0811 01:49:44.343104 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:44.343111 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:44.344673 (+ 1562us) server_negotiation.cc:407] Connection header received
0811 01:49:44.347745 (+ 3072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:44.347749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:44.347841 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:44.347950 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:44.349544 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:44.350263 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:44.352891 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:44.353387 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:44.353493 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:44.355310 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:44.355315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:44.355319 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:44.355352 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:44.355399 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:44.355412 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:44.355494 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:44.355801 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 01:49:44.355892 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":339,"threads_started":1}
I20260811 01:49:44.357822 13157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:44.343282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:43969 (local address 127.21.214.65:50329)
0811 01:49:44.344481 (+ 1199us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:44.344548 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:44.347358 (+ 2810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:44.348270 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:44.348285 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:44.348418 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:49:44.349340 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:44.349370 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:44.350452 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:44.350460 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:44.352710 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:44.352729 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:44.353637 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:44.353647 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:44.354008 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:44.354893 (+ 885us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:49:44.354900 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:49:44.354960 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:49:44.355074 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:49:44.355528 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:49:44.355543 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:49:44.355554 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 01:49:44.355776 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 01:49:44.356254 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":329,"threads_started":1}
I20260811 01:49:44.359851 13153 heartbeater.cc:347] Connected to a master server at 127.21.214.126:43969
I20260811 01:49:44.360185 13153 heartbeater.cc:464] Registering TS with master...
I20260811 01:49:44.361025 13153 heartbeater.cc:511] Master 127.21.214.126:43969 requested a full tablet report, sending...
I20260811 01:49:44.363430 12930 ts_manager.cc:194] Registered new tserver with Master: 1e70d921f7fb42a98530a0cc4d8d9ea9 (127.21.214.65:39587)
I20260811 01:49:44.365139 12930 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.214.65:50329
W20260811 01:49:44.742202 13158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:44.742672 13158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:44.742858 13158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:44.753623 13158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:44.753770 13158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:44.753906 13158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.66
I20260811 01:49:44.765401 13158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:35803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.214.66
--webserver_port=0
--tserver_master_addrs=127.21.214.126:43969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.66
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:44.767059 13158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:44.769376 13158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:44.785552 13166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:44.785473 13165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:44.785508 13168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:44.786984 13158 server_base.cc:1034] running on GCE node
I20260811 01:49:44.787709 13158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:44.788866 13158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:44.790130 13158 hybrid_clock.cc:648] HybridClock initialized: now 1786412984790035 us; error 91 us; skew 500 ppm
I20260811 01:49:44.790496 13158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:49:44.793574 13158 webserver.cc:460] Webserver started at http://127.21.214.66:35867/ using document root <none> and password file <none>
I20260811 01:49:44.794432 13158 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:44.794585 13158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:44.794951 13158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:44.798484 13158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-1/data/instance:
uuid: "fac8fb4d72a74b87a74cc1aac3724918"
format_stamp: "Formatted at 2026-08-11 01:49:44 on dist-test-slave-9gcw"
server_key: "4d645b1cb9ccbb3648e2234f1348c0fc"
server_key_iv: "5df143a3366c3ae39ab0680da7cfb98a"
server_key_version: "encryptionkey@0"
I20260811 01:49:44.799415 13158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance:
uuid: "fac8fb4d72a74b87a74cc1aac3724918"
format_stamp: "Formatted at 2026-08-11 01:49:44 on dist-test-slave-9gcw"
server_key: "4d645b1cb9ccbb3648e2234f1348c0fc"
server_key_iv: "5df143a3366c3ae39ab0680da7cfb98a"
server_key_version: "encryptionkey@0"
I20260811 01:49:44.805821 13158 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 01:49:44.810283 13174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:44.811877 13158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260811 01:49:44.812112 13158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-1/wal
uuid: "fac8fb4d72a74b87a74cc1aac3724918"
format_stamp: "Formatted at 2026-08-11 01:49:44 on dist-test-slave-9gcw"
server_key: "4d645b1cb9ccbb3648e2234f1348c0fc"
server_key_iv: "5df143a3366c3ae39ab0680da7cfb98a"
server_key_version: "encryptionkey@0"
I20260811 01:49:44.812378 13158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:44.824433 13158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:44.825433 13158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:44.825783 13158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:44.826870 13158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:44.828467 13158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:44.828610 13158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:44.828749 13158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:44.828840 13158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:44.865659 13158 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.66:46307
I20260811 01:49:44.865728 13301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.66:46307 every 8 connection(s)
I20260811 01:49:44.867255 13158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-1/data/info.pb
I20260811 01:49:44.873884 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 13158
I20260811 01:49:44.874096 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-1/wal/instance
I20260811 01:49:44.874747 22361 external_mini_cluster.cc:1428] Setting key 674e713693e6911c62c809653962ead6
I20260811 01:49:44.877909 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.214.67:0
--local_ip_for_outbound_sockets=127.21.214.67
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:43969
--builtin_ntp_servers=127.21.214.84:35803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 01:49:44.882112 13304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:44.870277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.66:50085 (local address 127.21.214.126:43969)
0811 01:49:44.870812 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:44.870819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:44.872105 (+ 1286us) server_negotiation.cc:407] Connection header received
0811 01:49:44.874349 (+ 2244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:44.874353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:44.874427 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:44.874532 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:44.875930 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:44.876851 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:44.878722 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:44.879144 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:44.879297 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:44.881096 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:44.881101 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:44.881164 (+ 63us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:44.881192 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:44.881257 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:44.881290 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:44.881397 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:44.881699 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 01:49:44.881830 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":232,"threads_started":1}
I20260811 01:49:44.883769 13305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:44.870930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:43969 (local address 127.21.214.66:50085)
0811 01:49:44.871930 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:44.871973 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:44.873978 (+ 2005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:44.874852 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:44.874864 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:44.874999 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:49:44.875758 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:44.875777 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:44.877012 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:44.877018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:44.878560 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:44.878574 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:44.879407 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:44.879415 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:44.879671 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:44.880608 (+ 937us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:49:44.880611 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:49:44.880659 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:49:44.880833 (+ 174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:49:44.881423 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:49:44.881438 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:49:44.881451 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 01:49:44.881687 (+ 236us) client_negotiation.cc:240] Negotiation successful
0811 01:49:44.882211 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":294,"threads_started":1}
I20260811 01:49:44.885596 13302 heartbeater.cc:347] Connected to a master server at 127.21.214.126:43969
I20260811 01:49:44.885974 13302 heartbeater.cc:464] Registering TS with master...
I20260811 01:49:44.886996 13302 heartbeater.cc:511] Master 127.21.214.126:43969 requested a full tablet report, sending...
I20260811 01:49:44.888866 12930 ts_manager.cc:194] Registered new tserver with Master: fac8fb4d72a74b87a74cc1aac3724918 (127.21.214.66:46307)
I20260811 01:49:44.890139 12930 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.214.66:50085
W20260811 01:49:45.251897 13306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:45.252210 13306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:45.252377 13306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:45.260093 13306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:45.260252 13306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:45.260399 13306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.214.67
I20260811 01:49:45.270275 13306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:35803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.214.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.214.67
--webserver_port=0
--tserver_master_addrs=127.21.214.126:43969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.214.67
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:45.271656 13306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:45.273672 13306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:45.288239 13311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:45.288136 13316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:45.288129 13313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:45.289006 13306 server_base.cc:1034] running on GCE node
I20260811 01:49:45.289696 13306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:45.290742 13306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:45.291960 13306 hybrid_clock.cc:648] HybridClock initialized: now 1786412985291905 us; error 41 us; skew 500 ppm
I20260811 01:49:45.292317 13306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 01:49:45.294723 13306 webserver.cc:460] Webserver started at http://127.21.214.67:38573/ using document root <none> and password file <none>
I20260811 01:49:45.295490 13306 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:45.295612 13306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:45.295964 13306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:45.299124 13306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-2/data/instance:
uuid: "7cdc8b74e74e4f0287cdfcbb2e6f150b"
format_stamp: "Formatted at 2026-08-11 01:49:45 on dist-test-slave-9gcw"
server_key: "2f1b72ffb51556025afa3ee569ef33d3"
server_key_iv: "f3076e73f6a28e95e2924a44aa62c76c"
server_key_version: "encryptionkey@0"
I20260811 01:49:45.299981 13306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance:
uuid: "7cdc8b74e74e4f0287cdfcbb2e6f150b"
format_stamp: "Formatted at 2026-08-11 01:49:45 on dist-test-slave-9gcw"
server_key: "2f1b72ffb51556025afa3ee569ef33d3"
server_key_iv: "f3076e73f6a28e95e2924a44aa62c76c"
server_key_version: "encryptionkey@0"
I20260811 01:49:45.305631 13306 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 01:49:45.309623 13325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:45.311136 13306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:49:45.311337 13306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-2/wal
uuid: "7cdc8b74e74e4f0287cdfcbb2e6f150b"
format_stamp: "Formatted at 2026-08-11 01:49:45 on dist-test-slave-9gcw"
server_key: "2f1b72ffb51556025afa3ee569ef33d3"
server_key_iv: "f3076e73f6a28e95e2924a44aa62c76c"
server_key_version: "encryptionkey@0"
I20260811 01:49:45.311574 13306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:45.331630 13306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:45.332567 13306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:45.332904 13306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:45.334120 13306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 01:49:45.335758 13306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 01:49:45.335850 13306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:45.336019 13306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 01:49:45.336095 13306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:45.368870 13153 heartbeater.cc:503] Master 127.21.214.126:43969 was elected leader, sending a full tablet report...
I20260811 01:49:45.373109 13306 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.67:35319
I20260811 01:49:45.373241 13455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.67:35319 every 8 connection(s)
I20260811 01:49:45.374573 13306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-2/data/info.pb
I20260811 01:49:45.384918 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 13306
I20260811 01:49:45.385171 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-2/wal/instance
I20260811 01:49:45.385946 22361 external_mini_cluster.cc:1428] Setting key 053158d59f3f7c2870d014cf43c519f9
I20260811 01:49:45.389675 13304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:45.377469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.67:44385 (local address 127.21.214.126:43969)
0811 01:49:45.377698 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:45.377705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:45.379260 (+ 1555us) server_negotiation.cc:407] Connection header received
0811 01:49:45.381473 (+ 2213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:45.381478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:45.381553 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:45.381682 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:45.383002 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.383766 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.386345 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.386750 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.386866 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:45.388719 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:45.388726 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:45.388730 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:45.388758 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:45.388809 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:45.388826 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:45.388925 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:45.389273 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 01:49:45.389389 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 01:49:45.390959 13460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:45.378141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:43969 (local address 127.21.214.67:44385)
0811 01:49:45.379095 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:45.379140 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:45.381091 (+ 1951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:45.382031 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:45.382039 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:45.382136 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:49:45.382804 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:45.382822 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.383960 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.383964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:45.386176 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:45.386194 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.387013 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.387022 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:45.387317 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:45.388326 (+ 1009us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 01:49:45.388330 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:49:45.388393 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 01:49:45.388514 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:49:45.388941 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:49:45.388951 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:49:45.388957 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 01:49:45.389256 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 01:49:45.389757 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":290,"threads_started":1}
I20260811 01:49:45.392402 13457 heartbeater.cc:347] Connected to a master server at 127.21.214.126:43969
I20260811 01:49:45.392756 13457 heartbeater.cc:464] Registering TS with master...
I20260811 01:49:45.393728 13457 heartbeater.cc:511] Master 127.21.214.126:43969 requested a full tablet report, sending...
I20260811 01:49:45.395378 12930 ts_manager.cc:194] Registered new tserver with Master: 7cdc8b74e74e4f0287cdfcbb2e6f150b (127.21.214.67:35319)
I20260811 01:49:45.396171 12930 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.214.67:44385
I20260811 01:49:45.402215 22361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 01:49:45.418053 13304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:45.411537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48326 (local address 127.21.214.126:43969)
0811 01:49:45.411731 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:45.411737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:45.412218 (+ 481us) server_negotiation.cc:407] Connection header received
0811 01:49:45.412372 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:45.412376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:45.412447 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:45.412540 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:45.413135 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.413963 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.415584 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.416004 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.416142 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:45.417424 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:45.417429 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:45.417432 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:45.417452 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:45.417486 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:45.417499 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:45.417572 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:45.417690 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 01:49:45.417784 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 01:49:45.422127 12930 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 01:49:45.429111 12930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 01:49:45.467418 13475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:45.443755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56008 (local address 127.21.214.65:39587)
0811 01:49:45.452127 (+ 8372us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:45.452141 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:45.457692 (+ 5551us) server_negotiation.cc:407] Connection header received
0811 01:49:45.457905 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:45.457918 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:45.458251 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:45.458489 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:45.460170 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.461550 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.464978 (+ 3428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.466069 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.466244 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:45.466582 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:45.466846 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 01:49:45.467041 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8091,"thread_start_us":7917,"threads_started":1}
I20260811 01:49:45.467907 13476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:45.444161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45448 (local address 127.21.214.66:46307)
0811 01:49:45.452658 (+ 8497us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:45.452670 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:45.458395 (+ 5725us) server_negotiation.cc:407] Connection header received
0811 01:49:45.458554 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:45.458565 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:45.458804 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:45.459056 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:45.459834 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.461505 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.464617 (+ 3112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.466244 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.466381 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:45.466628 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:45.467396 (+ 768us) server_negotiation.cc:299] Negotiation successful
0811 01:49:45.467587 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8287,"thread_start_us":8167,"threads_started":1}
I20260811 01:49:45.468259 13473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:45.442835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41204 (local address 127.21.214.67:35319)
0811 01:49:45.458994 (+ 16159us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:45.459003 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:45.459038 (+ 35us) server_negotiation.cc:407] Connection header received
0811 01:49:45.459130 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:45.459136 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:45.459358 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:45.459565 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:45.460558 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.462004 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.464562 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.466237 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.466381 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:45.466661 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:45.467625 (+ 964us) server_negotiation.cc:299] Negotiation successful
0811 01:49:45.467808 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15968,"thread_start_us":15789,"threads_started":1}
I20260811 01:49:45.468642 13472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:45.442502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:35319 (local address 127.0.0.1:41204)
0811 01:49:45.455438 (+ 12936us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:45.455516 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:45.455825 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:45.459895 (+ 4070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:45.459899 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:45.459925 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:45.460366 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:45.460378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.462164 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.462169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:45.464279 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:45.464301 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.466794 (+ 2493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.466802 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:45.467076 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:45.467163 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 01:49:45.467256 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 01:49:45.467395 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12486,"mutex_wait_us":66,"thread_start_us":14702,"threads_started":1}
I20260811 01:49:45.468797 13474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:45.443113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:46307 (local address 127.0.0.1:45448)
0811 01:49:45.458243 (+ 15130us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:45.458264 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:45.458391 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:45.459033 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:45.459038 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:45.459066 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:45.459585 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:45.459598 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.461657 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.461663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:45.464363 (+ 2700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:45.464380 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.466794 (+ 2414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.466802 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:45.467083 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:45.467161 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 01:49:45.468217 (+ 1056us) client_negotiation.cc:240] Negotiation successful
0811 01:49:45.468389 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14997,"mutex_wait_us":172,"thread_start_us":14877,"threads_started":1}
I20260811 01:49:45.467535 13471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:45.442669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.65:39587 (local address 127.0.0.1:56008)
0811 01:49:45.457521 (+ 14852us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:45.457538 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:45.457713 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:45.458841 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:45.458848 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:45.458881 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:45.459896 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:45.459908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.461737 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.461742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:45.464741 (+ 2999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:45.464752 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.466290 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.466296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:45.466547 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:45.466615 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 01:49:45.466807 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 01:49:45.467034 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14711,"mutex_wait_us":347,"thread_start_us":12136,"threads_started":1}
I20260811 01:49:45.472563 13382 tablet_service.cc:1467] Processing CreateTablet for tablet 92f71dbfb1eb44c48e2f1a264e0c5f8f (DEFAULT_TABLE table=test-table [id=a92c83d9addd4885b6d6692fcf0b37f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:49:45.472779 13229 tablet_service.cc:1467] Processing CreateTablet for tablet 92f71dbfb1eb44c48e2f1a264e0c5f8f (DEFAULT_TABLE table=test-table [id=a92c83d9addd4885b6d6692fcf0b37f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:49:45.472769 13076 tablet_service.cc:1467] Processing CreateTablet for tablet 92f71dbfb1eb44c48e2f1a264e0c5f8f (DEFAULT_TABLE table=test-table [id=a92c83d9addd4885b6d6692fcf0b37f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 01:49:45.474208 13076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92f71dbfb1eb44c48e2f1a264e0c5f8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:45.474215 13382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92f71dbfb1eb44c48e2f1a264e0c5f8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:45.474596 13229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92f71dbfb1eb44c48e2f1a264e0c5f8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:45.488579 13477 tablet_bootstrap.cc:492] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b: Bootstrap starting.
I20260811 01:49:45.490180 13479 tablet_bootstrap.cc:492] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918: Bootstrap starting.
I20260811 01:49:45.492823 13477 tablet_bootstrap.cc:654] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:45.493988 13479 tablet_bootstrap.cc:654] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:45.494297 13477 log.cc:824] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:45.495376 13479 log.cc:824] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:45.496724 13478 tablet_bootstrap.cc:492] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9: Bootstrap starting.
I20260811 01:49:45.498231 13477 tablet_bootstrap.cc:492] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b: No bootstrap required, opened a new log
I20260811 01:49:45.498541 13477 ts_tablet_manager.cc:1397] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260811 01:49:45.501940 13478 tablet_bootstrap.cc:654] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:45.503345 13478 log.cc:824] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:45.504356 13479 tablet_bootstrap.cc:492] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918: No bootstrap required, opened a new log
I20260811 01:49:45.504676 13479 ts_tablet_manager.cc:1397] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.006s
I20260811 01:49:45.506680 13478 tablet_bootstrap.cc:492] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9: No bootstrap required, opened a new log
I20260811 01:49:45.506958 13478 ts_tablet_manager.cc:1397] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 01:49:45.506887 13477 raft_consensus.cc:348] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdc8b74e74e4f0287cdfcbb2e6f150b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35319 } } peers { permanent_uuid: "1e70d921f7fb42a98530a0cc4d8d9ea9" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39587 } } peers { permanent_uuid: "fac8fb4d72a74b87a74cc1aac3724918" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46307 } }
I20260811 01:49:45.507341 13477 raft_consensus.cc:374] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:45.507483 13477 raft_consensus.cc:729] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdc8b74e74e4f0287cdfcbb2e6f150b, State: Initialized, Role: FOLLOWER
I20260811 01:49:45.508176 13477 consensus_queue.cc:260] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdc8b74e74e4f0287cdfcbb2e6f150b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35319 } } peers { permanent_uuid: "1e70d921f7fb42a98530a0cc4d8d9ea9" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39587 } } peers { permanent_uuid: "fac8fb4d72a74b87a74cc1aac3724918" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46307 } }
I20260811 01:49:45.515612 13457 heartbeater.cc:503] Master 127.21.214.126:43969 was elected leader, sending a full tablet report...
I20260811 01:49:45.515810 13479 raft_consensus.cc:348] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdc8b74e74e4f0287cdfcbb2e6f150b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35319 } } peers { permanent_uuid: "1e70d921f7fb42a98530a0cc4d8d9ea9" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39587 } } peers { permanent_uuid: "fac8fb4d72a74b87a74cc1aac3724918" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46307 } }
I20260811 01:49:45.516317 13479 raft_consensus.cc:374] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:45.516431 13479 raft_consensus.cc:729] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fac8fb4d72a74b87a74cc1aac3724918, State: Initialized, Role: FOLLOWER
I20260811 01:49:45.516568 13477 ts_tablet_manager.cc:1428] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b: Time spent starting tablet: real 0.018s user 0.015s sys 0.004s
I20260811 01:49:45.517122 13479 consensus_queue.cc:260] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdc8b74e74e4f0287cdfcbb2e6f150b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35319 } } peers { permanent_uuid: "1e70d921f7fb42a98530a0cc4d8d9ea9" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39587 } } peers { permanent_uuid: "fac8fb4d72a74b87a74cc1aac3724918" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46307 } }
I20260811 01:49:45.518433 13302 heartbeater.cc:503] Master 127.21.214.126:43969 was elected leader, sending a full tablet report...
I20260811 01:49:45.519142 13479 ts_tablet_manager.cc:1428] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918: Time spent starting tablet: real 0.014s user 0.010s sys 0.006s
I20260811 01:49:45.519559 13478 raft_consensus.cc:348] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdc8b74e74e4f0287cdfcbb2e6f150b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35319 } } peers { permanent_uuid: "1e70d921f7fb42a98530a0cc4d8d9ea9" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39587 } } peers { permanent_uuid: "fac8fb4d72a74b87a74cc1aac3724918" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46307 } }
I20260811 01:49:45.520080 13478 raft_consensus.cc:374] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:45.520190 13478 raft_consensus.cc:729] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e70d921f7fb42a98530a0cc4d8d9ea9, State: Initialized, Role: FOLLOWER
I20260811 01:49:45.520911 13478 consensus_queue.cc:260] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdc8b74e74e4f0287cdfcbb2e6f150b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35319 } } peers { permanent_uuid: "1e70d921f7fb42a98530a0cc4d8d9ea9" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39587 } } peers { permanent_uuid: "fac8fb4d72a74b87a74cc1aac3724918" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46307 } }
I20260811 01:49:45.522887 13478 ts_tablet_manager.cc:1428] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
W20260811 01:49:45.592170 13154 tablet.cc:2369] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:49:45.619609 13303 tablet.cc:2369] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 01:49:45.626454 13458 tablet.cc:2369] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:49:45.806037 13485 raft_consensus.cc:482] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 01:49:45.806499 13485 raft_consensus.cc:504] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdc8b74e74e4f0287cdfcbb2e6f150b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35319 } } peers { permanent_uuid: "1e70d921f7fb42a98530a0cc4d8d9ea9" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39587 } } peers { permanent_uuid: "fac8fb4d72a74b87a74cc1aac3724918" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46307 } }
I20260811 01:49:45.808463 13485 leader_election.cc:288] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cdc8b74e74e4f0287cdfcbb2e6f150b (127.21.214.67:35319), fac8fb4d72a74b87a74cc1aac3724918 (127.21.214.66:46307)
I20260811 01:49:45.815546 13473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:45.809194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:55931 (local address 127.21.214.67:35319)
0811 01:49:45.809419 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:45.809427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:45.810208 (+ 781us) server_negotiation.cc:407] Connection header received
0811 01:49:45.810461 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:45.810465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:45.810526 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:45.810623 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:45.811205 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.811904 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.813850 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.814578 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.814757 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:45.814809 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:45.815134 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 01:49:45.815285 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 01:49:45.815585 13486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:45.809020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.67:35319 (local address 127.21.214.65:55931)
0811 01:49:45.810055 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:45.810076 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:45.810288 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:45.810670 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:45.810675 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:45.810714 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:45.811046 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:45.811055 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.812044 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.812052 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:45.813628 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:45.813643 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.814789 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.814795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:45.814961 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:45.814998 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 01:49:45.815103 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 01:49:45.815255 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"thread_start_us":611,"threads_started":1}
I20260811 01:49:45.816340 13487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:45.809079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.66:46307 (local address 127.21.214.65:46293)
0811 01:49:45.810520 (+ 1441us) negotiation.cc:107] Waiting for socket to connect
0811 01:49:45.810536 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 01:49:45.810701 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:49:45.811062 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:49:45.811066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:49:45.811089 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 01:49:45.811408 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:45.811416 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.812514 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.812519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:45.813990 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:49:45.814002 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.815318 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.815384 (+ 66us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:49:45.815554 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:45.815601 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 01:49:45.815810 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 01:49:45.816006 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1327,"spinlock_wait_cycles":68352,"thread_start_us":1070,"threads_started":1}
I20260811 01:49:45.816483 13476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:45.809461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.214.65:46293 (local address 127.21.214.66:46307)
0811 01:49:45.809842 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:45.809852 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:45.810656 (+ 804us) server_negotiation.cc:407] Connection header received
0811 01:49:45.810861 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:45.810870 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:45.810940 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:45.811042 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 01:49:45.811574 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.812354 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.814175 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.815128 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.815291 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:45.815372 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:45.816023 (+ 651us) server_negotiation.cc:299] Negotiation successful
0811 01:49:45.816158 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 01:49:45.816901 13404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92f71dbfb1eb44c48e2f1a264e0c5f8f" candidate_uuid: "1e70d921f7fb42a98530a0cc4d8d9ea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdc8b74e74e4f0287cdfcbb2e6f150b" is_pre_election: true
I20260811 01:49:45.817518 13404 raft_consensus.cc:2454] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e70d921f7fb42a98530a0cc4d8d9ea9 in term 0.
I20260811 01:49:45.817790 13250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92f71dbfb1eb44c48e2f1a264e0c5f8f" candidate_uuid: "1e70d921f7fb42a98530a0cc4d8d9ea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fac8fb4d72a74b87a74cc1aac3724918" is_pre_election: true
I20260811 01:49:45.818225 13020 leader_election.cc:302] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e70d921f7fb42a98530a0cc4d8d9ea9, 7cdc8b74e74e4f0287cdfcbb2e6f150b; no voters:
I20260811 01:49:45.818308 13250 raft_consensus.cc:2454] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e70d921f7fb42a98530a0cc4d8d9ea9 in term 0.
I20260811 01:49:45.818615 13485 raft_consensus.cc:2781] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 01:49:45.818813 13485 raft_consensus.cc:482] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 01:49:45.818948 13485 raft_consensus.cc:3037] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:45.821310 13485 raft_consensus.cc:504] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdc8b74e74e4f0287cdfcbb2e6f150b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35319 } } peers { permanent_uuid: "1e70d921f7fb42a98530a0cc4d8d9ea9" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39587 } } peers { permanent_uuid: "fac8fb4d72a74b87a74cc1aac3724918" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46307 } }
I20260811 01:49:45.822000 13485 leader_election.cc:288] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [CANDIDATE]: Term 1 election: Requested vote from peers 7cdc8b74e74e4f0287cdfcbb2e6f150b (127.21.214.67:35319), fac8fb4d72a74b87a74cc1aac3724918 (127.21.214.66:46307)
I20260811 01:49:45.822402 13404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92f71dbfb1eb44c48e2f1a264e0c5f8f" candidate_uuid: "1e70d921f7fb42a98530a0cc4d8d9ea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cdc8b74e74e4f0287cdfcbb2e6f150b"
I20260811 01:49:45.822490 13250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92f71dbfb1eb44c48e2f1a264e0c5f8f" candidate_uuid: "1e70d921f7fb42a98530a0cc4d8d9ea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fac8fb4d72a74b87a74cc1aac3724918"
I20260811 01:49:45.822597 13404 raft_consensus.cc:3037] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:45.822732 13250 raft_consensus.cc:3037] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:45.824939 13404 raft_consensus.cc:2454] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e70d921f7fb42a98530a0cc4d8d9ea9 in term 1.
I20260811 01:49:45.824975 13250 raft_consensus.cc:2454] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e70d921f7fb42a98530a0cc4d8d9ea9 in term 1.
I20260811 01:49:45.825389 13020 leader_election.cc:302] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e70d921f7fb42a98530a0cc4d8d9ea9, 7cdc8b74e74e4f0287cdfcbb2e6f150b; no voters:
I20260811 01:49:45.825717 13485 raft_consensus.cc:2781] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:49:45.826236 13485 raft_consensus.cc:686] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [term 1 LEADER]: Becoming Leader. State: Replica: 1e70d921f7fb42a98530a0cc4d8d9ea9, State: Running, Role: LEADER
I20260811 01:49:45.826776 13485 consensus_queue.cc:237] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdc8b74e74e4f0287cdfcbb2e6f150b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35319 } } peers { permanent_uuid: "1e70d921f7fb42a98530a0cc4d8d9ea9" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39587 } } peers { permanent_uuid: "fac8fb4d72a74b87a74cc1aac3724918" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46307 } }
I20260811 01:49:45.831034 12929 catalog_manager.cc:5495] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e70d921f7fb42a98530a0cc4d8d9ea9 (127.21.214.65). New cstate: current_term: 1 leader_uuid: "1e70d921f7fb42a98530a0cc4d8d9ea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cdc8b74e74e4f0287cdfcbb2e6f150b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e70d921f7fb42a98530a0cc4d8d9ea9" member_type: VOTER last_known_addr { host: "127.21.214.65" port: 39587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fac8fb4d72a74b87a74cc1aac3724918" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46307 } health_report { overall_health: UNKNOWN } } }
I20260811 01:49:45.929761 13475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:45.923343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56016 (local address 127.21.214.65:39587)
0811 01:49:45.923588 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:45.923595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:45.923787 (+ 192us) server_negotiation.cc:407] Connection header received
0811 01:49:45.924132 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:45.924137 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:45.924239 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:45.924415 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:49:45.924887 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.925712 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.927262 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.927730 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.927888 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:45.928471 (+ 583us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:49:45.929019 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:49:45.929224 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:45.929381 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 01:49:45.929468 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 01:49:45.938381 13250 raft_consensus.cc:1261] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918 [term 1 FOLLOWER]: Refusing update from remote peer 1e70d921f7fb42a98530a0cc4d8d9ea9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:49:45.938558 13404 raft_consensus.cc:1261] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b [term 1 FOLLOWER]: Refusing update from remote peer 1e70d921f7fb42a98530a0cc4d8d9ea9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:49:45.939465 13488 consensus_queue.cc:1035] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fac8fb4d72a74b87a74cc1aac3724918" member_type: VOTER last_known_addr { host: "127.21.214.66" port: 46307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:49:45.939918 13485 consensus_queue.cc:1035] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdc8b74e74e4f0287cdfcbb2e6f150b" member_type: VOTER last_known_addr { host: "127.21.214.67" port: 35319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:49:45.953804 13492 mvcc.cc:205] Tried to move back new op lower bound from 7317147590393704448 to 7317147589950586880. Current Snapshot: MvccSnapshot[applied={T|T < 7317147590393704448}]
I20260811 01:49:45.966858 12930 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:49:45.967190 12930 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 01:49:45.971503 12930 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa55308b13c84e83ae45ff025e078afb: Sending DeleteTablet for 3 replicas of tablet 92f71dbfb1eb44c48e2f1a264e0c5f8f
I20260811 01:49:45.972868 13382 tablet_service.cc:1514] Processing DeleteTablet for tablet 92f71dbfb1eb44c48e2f1a264e0c5f8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:45 UTC) from {username='slave'} at 127.0.0.1:41204
I20260811 01:49:45.972927 13076 tablet_service.cc:1514] Processing DeleteTablet for tablet 92f71dbfb1eb44c48e2f1a264e0c5f8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:45 UTC) from {username='slave'} at 127.0.0.1:56008
I20260811 01:49:45.973093 13229 tablet_service.cc:1514] Processing DeleteTablet for tablet 92f71dbfb1eb44c48e2f1a264e0c5f8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 01:49:45 UTC) from {username='slave'} at 127.0.0.1:45448
I20260811 01:49:45.978863 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 13002
I20260811 01:49:45.985492 13505 tablet_replica.cc:330] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918: stopping tablet replica
I20260811 01:49:45.986032 13505 raft_consensus.cc:2229] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:49:45.986661 13505 raft_consensus.cc:2258] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:49:45.987613 13503 tablet_replica.cc:330] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9: stopping tablet replica
I20260811 01:49:45.988050 13503 raft_consensus.cc:2229] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 01:49:45.988756 13503 raft_consensus.cc:2258] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:49:45.991101 13505 ts_tablet_manager.cc:1905] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:49:45.996833 13503 ts_tablet_manager.cc:1905] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:49:46.000794 13505 ts_tablet_manager.cc:1918] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:49:46.001026 13505 log.cc:1196] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-1/wal/wals/92f71dbfb1eb44c48e2f1a264e0c5f8f
I20260811 01:49:46.001643 13505 ts_tablet_manager.cc:1939] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P fac8fb4d72a74b87a74cc1aac3724918: Deleting consensus metadata
I20260811 01:49:46.002559 13504 tablet_replica.cc:330] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b: stopping tablet replica
I20260811 01:49:46.004441 13503 ts_tablet_manager.cc:1918] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:49:46.004657 13503 log.cc:1196] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-0/wal/wals/92f71dbfb1eb44c48e2f1a264e0c5f8f
I20260811 01:49:46.005350 13503 ts_tablet_manager.cc:1939] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 1e70d921f7fb42a98530a0cc4d8d9ea9: Deleting consensus metadata
I20260811 01:49:46.006153 12912 catalog_manager.cc:4841] TS fac8fb4d72a74b87a74cc1aac3724918 (127.21.214.66:46307): tablet 92f71dbfb1eb44c48e2f1a264e0c5f8f (table test-table [id=a92c83d9addd4885b6d6692fcf0b37f2]) successfully deleted
I20260811 01:49:46.007305 13504 raft_consensus.cc:2229] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 01:49:46.007391 12914 catalog_manager.cc:4841] TS 1e70d921f7fb42a98530a0cc4d8d9ea9 (127.21.214.65:39587): tablet 92f71dbfb1eb44c48e2f1a264e0c5f8f (table test-table [id=a92c83d9addd4885b6d6692fcf0b37f2]) successfully deleted
I20260811 01:49:46.007838 13504 raft_consensus.cc:2258] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 01:49:46.008201 13475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:45.980780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56018 (local address 127.21.214.65:39587)
0811 01:49:45.983056 (+ 2276us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:45.983065 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:45.993021 (+ 9956us) server_negotiation.cc:407] Connection header received
0811 01:49:45.993626 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:45.993636 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:45.993784 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:45.993883 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:45.994853 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:45.996499 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:45.999379 (+ 2880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:46.000314 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:46.000444 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:46.003250 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:46.003294 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:46.003315 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:46.006917 (+ 3602us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:46.006984 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:46.007019 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:46.007142 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:46.007671 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 01:49:46.007807 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2099}
I20260811 01:49:46.009470 13147 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:46.010553 13504 ts_tablet_manager.cc:1905] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 01:49:46.018965 13504 ts_tablet_manager.cc:1918] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 01:49:46.019171 13504 log.cc:1196] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b: Deleting WAL directory at /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786412636131983-22361-0/minicluster-data/ts-2/wal/wals/92f71dbfb1eb44c48e2f1a264e0c5f8f
I20260811 01:49:46.019685 13504 ts_tablet_manager.cc:1939] T 92f71dbfb1eb44c48e2f1a264e0c5f8f P 7cdc8b74e74e4f0287cdfcbb2e6f150b: Deleting consensus metadata
I20260811 01:49:46.021904 12911 catalog_manager.cc:4841] TS 7cdc8b74e74e4f0287cdfcbb2e6f150b (127.21.214.67:35319): tablet 92f71dbfb1eb44c48e2f1a264e0c5f8f (table test-table [id=a92c83d9addd4885b6d6692fcf0b37f2]) successfully deleted
I20260811 01:49:46.111557 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 13002
I20260811 01:49:46.136382 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 13158
I20260811 01:49:46.144084 13476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:46.137405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45456 (local address 127.21.214.66:46307)
0811 01:49:46.137614 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:46.137619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:46.137642 (+ 23us) server_negotiation.cc:407] Connection header received
0811 01:49:46.137749 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:46.137753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:46.137841 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:46.137950 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:46.138369 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:46.139261 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:46.140577 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:46.141046 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:46.141325 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:46.142355 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:46.142365 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:46.142382 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:46.143051 (+ 669us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:46.143104 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:46.143126 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:46.143245 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:46.143417 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 01:49:46.143697 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 01:49:46.144935 13296 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:46.240629 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 13158
I20260811 01:49:46.263316 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 13306
I20260811 01:49:46.270756 13473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:46.264341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41220 (local address 127.21.214.67:35319)
0811 01:49:46.264555 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:46.264561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:46.264581 (+ 20us) server_negotiation.cc:407] Connection header received
0811 01:49:46.264712 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:46.264717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:46.264823 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:46.264920 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:46.265437 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:46.266241 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:46.267600 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:46.268126 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:46.268238 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:46.269479 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:46.269488 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:46.269504 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:46.270090 (+ 586us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:46.270139 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:46.270159 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:46.270261 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:46.270360 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 01:49:46.270452 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 01:49:46.271492 13450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:46.364943 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 13306
I20260811 01:49:46.387254 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 12892
I20260811 01:49:46.388538 12971 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:46.501784 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 12892
2026-08-11T01:49:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3383 ms)
[----------] 6 tests from EncryptionPolicyTest (24516 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:49:46 dist-test-slave-9gcw krb5kdc[13515](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:49:46 dist-test-slave-9gcw krb5kdc[13515](info): set up 2 sockets
Aug 11 01:49:46 dist-test-slave-9gcw krb5kdc[13515](info): commencing operation
krb5kdc: starting...
W20260811 01:49:47.840664 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.001s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:49:48 dist-test-slave-9gcw krb5kdc[13515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:49:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:49:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 01:49:48.202497 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34881
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:49:48.567713 13531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:48.568032 13531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:48.568197 13531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:48.575661 13531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:49:48.575798 13531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:48.575850 13531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:48.575977 13531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:49:48.576040 13531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:49:48.586411 13531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34881
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:48.587877 13531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:48.589359 13531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:48.602525 13537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:48.602643 13538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:48.603011 13540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:48.603613 13531 server_base.cc:1034] running on GCE node
I20260811 01:49:48.604441 13531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:48.606053 13531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:48.607297 13531 hybrid_clock.cc:648] HybridClock initialized: now 1786412988607249 us; error 72 us; skew 500 ppm
I20260811 01:49:48.607666 13531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:48 dist-test-slave-9gcw krb5kdc[13515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412988, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:48.614423 13531 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 01:49:48.616158 13531 webserver.cc:460] Webserver started at http://127.0.0.1:37371/ using document root <none> and password file <none>
I20260811 01:49:48.616992 13531 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:48.617193 13531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:48.617547 13531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:48.620775 13531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "594e5ffada7641d483d4460c9c622c28"
format_stamp: "Formatted at 2026-08-11 01:49:48 on dist-test-slave-9gcw"
server_key: "937a609bf9da29b1a80fcbfb62e5d9b6"
server_key_iv: "37979eabccd5de26c2db1bd498410bd8"
server_key_version: "encryptionkey@0"
I20260811 01:49:48.621721 13531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "594e5ffada7641d483d4460c9c622c28"
format_stamp: "Formatted at 2026-08-11 01:49:48 on dist-test-slave-9gcw"
server_key: "937a609bf9da29b1a80fcbfb62e5d9b6"
server_key_iv: "37979eabccd5de26c2db1bd498410bd8"
server_key_version: "encryptionkey@0"
I20260811 01:49:48.627660 13531 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 01:49:48.631882 13547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:48.633914 13531 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 01:49:48.634150 13531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "594e5ffada7641d483d4460c9c622c28"
format_stamp: "Formatted at 2026-08-11 01:49:48 on dist-test-slave-9gcw"
server_key: "937a609bf9da29b1a80fcbfb62e5d9b6"
server_key_iv: "37979eabccd5de26c2db1bd498410bd8"
server_key_version: "encryptionkey@0"
I20260811 01:49:48.634393 13531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:48.653052 13531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:48.657684 13531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:48.658025 13531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:48.678520 13531 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34881
I20260811 01:49:48.678552 13606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34881 every 8 connection(s)
I20260811 01:49:48.680009 13531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:49:48.684412 13607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:48.687280 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 13531
I20260811 01:49:48.687536 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:49:48.688254 22361 external_mini_cluster.cc:1428] Setting key b9504ab1d3f0039b8225e1d148cff39c
I20260811 01:49:48.698004 13607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28: Bootstrap starting.
I20260811 01:49:48.702199 13607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:48.703691 13607 log.cc:824] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28: Log is configured to *not* fsync() on all Append() calls
Aug 11 01:49:48 dist-test-slave-9gcw krb5kdc[13515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 01:49:48.706995 13607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28: No bootstrap required, opened a new log
I20260811 01:49:48.713953 13611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:48.690476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59105 (local address 127.0.0.1:34881)
0811 01:49:48.691360 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:48.691373 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:48.691418 (+ 45us) server_negotiation.cc:407] Connection header received
0811 01:49:48.693267 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:48.693292 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:48.693773 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:48.694128 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:48.696164 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:48.697629 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:48.699495 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:48.700212 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:48.700347 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:48.705808 (+ 5461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:48.705816 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:48.705832 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:48.705869 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:48.709287 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:48.710180 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:48.710187 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:48.710204 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:48.710394 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:48.710919 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:48.710924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:48.710926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:48.711515 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:48.711893 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:48.712421 (+ 528us) server_negotiation.cc:299] Negotiation successful
0811 01:49:48.712766 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":414,"threads_started":1}
I20260811 01:49:48.715623 13607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594e5ffada7641d483d4460c9c622c28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34881 } }
I20260811 01:49:48.716043 13607 raft_consensus.cc:374] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:48.716176 13607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 594e5ffada7641d483d4460c9c622c28, State: Initialized, Role: FOLLOWER
I20260811 01:49:48.716895 13607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594e5ffada7641d483d4460c9c622c28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34881 } }
I20260811 01:49:48.717267 13607 raft_consensus.cc:388] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:49:48.717446 13607 raft_consensus.cc:482] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:49:48.717653 13607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:48.718384 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:49:48.720059 13607 raft_consensus.cc:504] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594e5ffada7641d483d4460c9c622c28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34881 } }
I20260811 01:49:48.720569 13607 leader_election.cc:302] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 594e5ffada7641d483d4460c9c622c28; no voters:
I20260811 01:49:48.721325 13607 leader_election.cc:288] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:49:48.721509 13613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:49:48.722267 13613 raft_consensus.cc:686] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28 [term 1 LEADER]: Becoming Leader. State: Replica: 594e5ffada7641d483d4460c9c622c28, State: Running, Role: LEADER
I20260811 01:49:48.723027 13613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594e5ffada7641d483d4460c9c622c28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34881 } }
I20260811 01:49:48.726496 13617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "594e5ffada7641d483d4460c9c622c28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594e5ffada7641d483d4460c9c622c28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34881 } } }
I20260811 01:49:48.726987 13617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28 [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:48.728166 13616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 594e5ffada7641d483d4460c9c622c28. Latest consensus state: current_term: 1 leader_uuid: "594e5ffada7641d483d4460c9c622c28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594e5ffada7641d483d4460c9c622c28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34881 } } }
I20260811 01:49:48.728540 13616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28 [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:48.731518 13607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:49:48.736877 13619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:49:48.744362 13619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:49:48.769001 13619 catalog_manager.cc:1347] Generated new cluster ID: 22ab282dbc724495a35dc28f20ad1111
I20260811 01:49:48.769165 13619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:49:48.773047 13611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:48.744434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57887 (local address 127.0.0.1:34881)
0811 01:49:48.744619 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:48.744625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:48.753626 (+ 9001us) server_negotiation.cc:407] Connection header received
0811 01:49:48.754183 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:48.754189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:48.754277 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:48.754356 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:48.756997 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:48.758263 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:48.760271 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:48.760982 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:48.761082 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:48.764950 (+ 3868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:48.764957 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:48.764963 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:48.765007 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:48.768962 (+ 3955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:48.770096 (+ 1134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:48.770103 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:48.770108 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:48.770274 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:48.770927 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:48.770935 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:48.770940 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:48.771400 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:48.771675 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:48.772458 (+ 783us) server_negotiation.cc:299] Negotiation successful
0811 01:49:48.772663 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 01:49:48.787669 13619 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:49:48.789018 13619 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:49:48.797910 13619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 594e5ffada7641d483d4460c9c622c28: Generated new TSK 0
I20260811 01:49:48.798812 13619 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:49:48.807610 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 13531
I20260811 01:49:48.809784 13600 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:48.906972 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 13531
W20260811 01:49:48.935132 13513 connection.cc:535] client connection to 127.0.0.1:34881 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34881): Connection reset by peer (error 104)
2026-08-11T01:49:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:49:49 dist-test-slave-9gcw krb5kdc[13649](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:49:49 dist-test-slave-9gcw krb5kdc[13649](info): set up 2 sockets
Aug 11 01:49:49 dist-test-slave-9gcw krb5kdc[13649](info): commencing operation
krb5kdc: starting...
W20260811 01:49:50.259716 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:49:50 dist-test-slave-9gcw krb5kdc[13649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:49:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:49:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 01:49:50.629725 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46489
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:49:50.993978 13666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:50.994306 13666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:50.994410 13666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:51.002512 13666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:49:51.002650 13666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:51.002705 13666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:51.002760 13666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:49:51.002797 13666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:49:51.013958 13666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46489
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:51.015388 13666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:51.016755 13666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:51.031355 13674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:51.031400 13676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:51.031301 13673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:51.032321 13666 server_base.cc:1034] running on GCE node
I20260811 01:49:51.033234 13666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:51.035132 13666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:51.036424 13666 hybrid_clock.cc:648] HybridClock initialized: now 1786412991036351 us; error 80 us; skew 500 ppm
I20260811 01:49:51.036870 13666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:51 dist-test-slave-9gcw krb5kdc[13649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412991, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:51.040730 13666 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 01:49:51.042719 13666 webserver.cc:460] Webserver started at http://127.0.0.1:43465/ using document root <none> and password file <none>
I20260811 01:49:51.043584 13666 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:51.043742 13666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:51.044144 13666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:51.047817 13666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "efac47d01d344398bd9af5e9297aa7e5"
format_stamp: "Formatted at 2026-08-11 01:49:51 on dist-test-slave-9gcw"
server_key: "87c9c3da265030d18197cfae2991e902"
server_key_iv: "8694e96e1e09db99ee00d63edf26f4ad"
server_key_version: "encryptionkey@0"
I20260811 01:49:51.048743 13666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "efac47d01d344398bd9af5e9297aa7e5"
format_stamp: "Formatted at 2026-08-11 01:49:51 on dist-test-slave-9gcw"
server_key: "87c9c3da265030d18197cfae2991e902"
server_key_iv: "8694e96e1e09db99ee00d63edf26f4ad"
server_key_version: "encryptionkey@0"
I20260811 01:49:51.054975 13666 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260811 01:49:51.059069 13685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:51.060637 13666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 01:49:51.060878 13666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "efac47d01d344398bd9af5e9297aa7e5"
format_stamp: "Formatted at 2026-08-11 01:49:51 on dist-test-slave-9gcw"
server_key: "87c9c3da265030d18197cfae2991e902"
server_key_iv: "8694e96e1e09db99ee00d63edf26f4ad"
server_key_version: "encryptionkey@0"
I20260811 01:49:51.061116 13666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:51.092285 13666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:51.097929 13666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:51.098357 13666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:51.122439 13666 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46489
I20260811 01:49:51.122471 13741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46489 every 8 connection(s)
I20260811 01:49:51.124291 13666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:49:51.126118 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 13666
I20260811 01:49:51.126310 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:49:51.126907 22361 external_mini_cluster.cc:1428] Setting key ade3e9f00c7a1afbabbde58403bbc328
I20260811 01:49:51.129534 13742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:51.143296 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:51.129057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59727 (local address 127.0.0.1:46489)
0811 01:49:51.130049 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:51.130062 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:51.130110 (+ 48us) server_negotiation.cc:407] Connection header received
0811 01:49:51.131969 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:51.131994 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:51.132469 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:51.132853 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:51.133596 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:51.135369 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:51.137537 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:51.138390 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:51.138547 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:51.140428 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:51.140437 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:51.140454 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:51.141298 (+ 844us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:51.141400 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:51.141430 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:51.141578 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:51.141945 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 01:49:51.142175 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":438,"threads_started":1}
I20260811 01:49:51.143965 13742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5: Bootstrap starting.
I20260811 01:49:51.148032 13742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:51.148288 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:49:51.149295 13742 log.cc:824] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:51.152763 13742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5: No bootstrap required, opened a new log
I20260811 01:49:51.161695 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:51.153692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45879 (local address 127.0.0.1:46489)
0811 01:49:51.153895 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:51.153902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:51.154537 (+ 635us) server_negotiation.cc:407] Connection header received
0811 01:49:51.154726 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:51.154736 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:51.154821 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:51.154954 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:51.155640 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:51.156937 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:51.158598 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:51.159345 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:51.159479 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:51.160761 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:51.160769 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:51.160779 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:51.160805 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:51.160863 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:51.160885 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:51.161006 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:51.161232 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 01:49:51.161306 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 01:49:51.163143 13742 raft_consensus.cc:348] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efac47d01d344398bd9af5e9297aa7e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46489 } }
I20260811 01:49:51.163659 13742 raft_consensus.cc:374] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:51.163815 13742 raft_consensus.cc:729] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efac47d01d344398bd9af5e9297aa7e5, State: Initialized, Role: FOLLOWER
I20260811 01:49:51.164429 13742 consensus_queue.cc:260] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efac47d01d344398bd9af5e9297aa7e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46489 } }
I20260811 01:49:51.164726 13742 raft_consensus.cc:388] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:49:51.164889 13742 raft_consensus.cc:482] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:49:51.165107 13742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:51.167835 13742 raft_consensus.cc:504] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efac47d01d344398bd9af5e9297aa7e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46489 } }
I20260811 01:49:51.168401 13742 leader_election.cc:302] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efac47d01d344398bd9af5e9297aa7e5; no voters:
I20260811 01:49:51.169034 13742 leader_election.cc:288] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:49:51.169250 13758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:49:51.170087 13758 raft_consensus.cc:686] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5 [term 1 LEADER]: Becoming Leader. State: Replica: efac47d01d344398bd9af5e9297aa7e5, State: Running, Role: LEADER
I20260811 01:49:51.170706 13758 consensus_queue.cc:237] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efac47d01d344398bd9af5e9297aa7e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46489 } }
I20260811 01:49:51.171444 13742 sys_catalog.cc:564] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:49:51.173979 13759 sys_catalog.cc:455] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efac47d01d344398bd9af5e9297aa7e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efac47d01d344398bd9af5e9297aa7e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46489 } } }
I20260811 01:49:51.174351 13759 sys_catalog.cc:458] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5 [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:51.175242 13761 sys_catalog.cc:455] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efac47d01d344398bd9af5e9297aa7e5. Latest consensus state: current_term: 1 leader_uuid: "efac47d01d344398bd9af5e9297aa7e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efac47d01d344398bd9af5e9297aa7e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46489 } } }
I20260811 01:49:51.175729 13761 sys_catalog.cc:458] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5 [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:51.176108 13764 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:49:51.182559 13764 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:49:51.192862 13764 catalog_manager.cc:1347] Generated new cluster ID: f8883df9df2b46ac95d2b09645c27d2f
I20260811 01:49:51.193005 13764 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:49:51.205881 13764 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:49:51.207476 13764 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:49:51.227324 13764 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efac47d01d344398bd9af5e9297aa7e5: Generated new TSK 0
I20260811 01:49:51.228504 13764 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:49:51.245625 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 13666
W20260811 01:49:51.247320 13737 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59727
W20260811 01:49:51.248307 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 13666: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:49:51.248514 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 13666
W20260811 01:49:51.272815 13647 connection.cc:535] client connection to 127.0.0.1:46489 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46489): Connection reset by peer (error 104)
2026-08-11T01:49:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:49:51 dist-test-slave-9gcw krb5kdc[13780](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:49:51 dist-test-slave-9gcw krb5kdc[13780](info): set up 2 sockets
Aug 11 01:49:51 dist-test-slave-9gcw krb5kdc[13780](info): commencing operation
krb5kdc: starting...
W20260811 01:49:52.534792 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:49:52 dist-test-slave-9gcw krb5kdc[13780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:49:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:49:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 01:49:52.884708 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38037
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:49:53.266750 13796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:53.267117 13796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:53.267293 13796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:53.275029 13796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:49:53.275167 13796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:53.275295 13796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:53.275427 13796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:49:53.275491 13796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:49:53.286175 13796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38037
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:53.287580 13796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:53.289091 13796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:53.307003 13801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:53.307366 13805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:53.307461 13803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:53.309937 13796 server_base.cc:1034] running on GCE node
I20260811 01:49:53.311051 13796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:53.312976 13796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:53.314234 13796 hybrid_clock.cc:648] HybridClock initialized: now 1786412993314186 us; error 87 us; skew 500 ppm
I20260811 01:49:53.314904 13796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:53 dist-test-slave-9gcw krb5kdc[13780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412993, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:53.320026 13796 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 01:49:53.321785 13796 webserver.cc:460] Webserver started at http://127.0.0.1:33121/ using document root <none> and password file <none>
I20260811 01:49:53.322656 13796 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:53.322840 13796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:53.323303 13796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:53.328423 13796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "30b8719f768f4dc293ad640f824526fb"
format_stamp: "Formatted at 2026-08-11 01:49:53 on dist-test-slave-9gcw"
server_key: "679a01a3ad054cd30394d33f1e7b14a4"
server_key_iv: "7993c20968763cf50ae6c88c6c82d67b"
server_key_version: "encryptionkey@0"
I20260811 01:49:53.329795 13796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "30b8719f768f4dc293ad640f824526fb"
format_stamp: "Formatted at 2026-08-11 01:49:53 on dist-test-slave-9gcw"
server_key: "679a01a3ad054cd30394d33f1e7b14a4"
server_key_iv: "7993c20968763cf50ae6c88c6c82d67b"
server_key_version: "encryptionkey@0"
I20260811 01:49:53.338605 13796 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260811 01:49:53.344638 13814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:53.346683 13796 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 01:49:53.346974 13796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "30b8719f768f4dc293ad640f824526fb"
format_stamp: "Formatted at 2026-08-11 01:49:53 on dist-test-slave-9gcw"
server_key: "679a01a3ad054cd30394d33f1e7b14a4"
server_key_iv: "7993c20968763cf50ae6c88c6c82d67b"
server_key_version: "encryptionkey@0"
I20260811 01:49:53.347287 13796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:53.357327 13796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:53.362172 13796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:53.362457 13796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:53.384143 13796 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38037
I20260811 01:49:53.384178 13876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38037 every 8 connection(s)
I20260811 01:49:53.385937 13796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:49:53.390398 13877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:53.390522 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 13796
I20260811 01:49:53.390743 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:49:53.391322 22361 external_mini_cluster.cc:1428] Setting key 4db02b89872f66f929bef91534513e8e
I20260811 01:49:53.401079 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:53.393627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45909 (local address 127.0.0.1:38037)
0811 01:49:53.394683 (+ 1056us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:53.394705 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:53.394764 (+ 59us) server_negotiation.cc:407] Connection header received
0811 01:49:53.397125 (+ 2361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:53.397213 (+ 88us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:53.397666 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:53.398036 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:53.399336 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:53.399344 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:53.399358 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:53.399374 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:53.399438 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:53.399464 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:53.399648 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:53.399788 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 01:49:53.400008 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753,"thread_start_us":506,"threads_started":1}
I20260811 01:49:53.405262 13877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb: Bootstrap starting.
I20260811 01:49:53.405436 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:49:53.410009 13877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:53.411609 13877 log.cc:824] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:53.414368 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:53.411434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46987 (local address 127.0.0.1:38037)
0811 01:49:53.411657 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:53.411663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:53.411950 (+ 287us) server_negotiation.cc:407] Connection header received
0811 01:49:53.412169 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:53.412188 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:53.412310 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:53.412456 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:53.413645 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:53.413650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:53.413653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:53.413669 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:53.413711 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:53.413729 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:53.413833 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:53.413986 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 01:49:53.414031 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 01:49:53.415558 13877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb: No bootstrap required, opened a new log
I20260811 01:49:53.423674 13877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30b8719f768f4dc293ad640f824526fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38037 } }
I20260811 01:49:53.424229 13877 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:53.424362 13877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30b8719f768f4dc293ad640f824526fb, State: Initialized, Role: FOLLOWER
I20260811 01:49:53.424993 13877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30b8719f768f4dc293ad640f824526fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38037 } }
I20260811 01:49:53.425288 13877 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:49:53.425421 13877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:49:53.425565 13877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:53.428041 13877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30b8719f768f4dc293ad640f824526fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38037 } }
I20260811 01:49:53.428593 13877 leader_election.cc:302] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30b8719f768f4dc293ad640f824526fb; no voters:
I20260811 01:49:53.429234 13877 leader_election.cc:288] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:49:53.429426 13887 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:49:53.430205 13887 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb [term 1 LEADER]: Becoming Leader. State: Replica: 30b8719f768f4dc293ad640f824526fb, State: Running, Role: LEADER
I20260811 01:49:53.430852 13887 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30b8719f768f4dc293ad640f824526fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38037 } }
I20260811 01:49:53.431665 13877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:49:53.434302 13890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30b8719f768f4dc293ad640f824526fb. Latest consensus state: current_term: 1 leader_uuid: "30b8719f768f4dc293ad640f824526fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30b8719f768f4dc293ad640f824526fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38037 } } }
I20260811 01:49:53.434474 13889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30b8719f768f4dc293ad640f824526fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30b8719f768f4dc293ad640f824526fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38037 } } }
I20260811 01:49:53.434773 13890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:53.434921 13889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:53.436022 13897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:49:53.442301 13897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:49:53.451841 13897 catalog_manager.cc:1347] Generated new cluster ID: 96ff9b612bc5407a84a2c74d804aa3c8
I20260811 01:49:53.451982 13897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:49:53.474606 13897 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:49:53.475123 13897 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:49:53.482331 13897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30b8719f768f4dc293ad640f824526fb: Generated new TSK 0
I20260811 01:49:53.483513 13897 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:49:53.489714 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 13796
W20260811 01:49:53.490981 13872 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45909
W20260811 01:49:53.491750 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 13796: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:49:53.491909 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 13796
2026-08-11T01:49:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2241 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:49:53 dist-test-slave-9gcw krb5kdc[13912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:49:53 dist-test-slave-9gcw krb5kdc[13912](info): set up 2 sockets
Aug 11 01:49:53 dist-test-slave-9gcw krb5kdc[13912](info): commencing operation
krb5kdc: starting...
W20260811 01:49:54.761273 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:49:54 dist-test-slave-9gcw krb5kdc[13912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:49:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:49:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 01:49:55.109606 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35739
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:49:55.470744 13931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:55.471067 13931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:55.471246 13931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:55.478785 13931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:49:55.478916 13931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:55.479043 13931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:55.479185 13931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:49:55.479259 13931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:49:55.489713 13931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:55.491185 13931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:55.492591 13931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:55.505352 13937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:55.505300 13941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:55.505386 13938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:55.506203 13931 server_base.cc:1034] running on GCE node
I20260811 01:49:55.507036 13931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:55.508610 13931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:55.509855 13931 hybrid_clock.cc:648] HybridClock initialized: now 1786412995509806 us; error 40 us; skew 500 ppm
I20260811 01:49:55.510246 13931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:55 dist-test-slave-9gcw krb5kdc[13912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412995, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:55.518611 13931 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 01:49:55.520454 13931 webserver.cc:460] Webserver started at http://127.0.0.1:46733/ using document root <none> and password file <none>
I20260811 01:49:55.521294 13931 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:55.521452 13931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:55.521795 13931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:55.525102 13931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "ac5e033f0ccf462b926edc986a870040"
format_stamp: "Formatted at 2026-08-11 01:49:55 on dist-test-slave-9gcw"
server_key: "76660cc0301a69631d8c72dcf7e113c1"
server_key_iv: "990a1ece02da707d12b941580cc4e90c"
server_key_version: "encryptionkey@0"
I20260811 01:49:55.526039 13931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "ac5e033f0ccf462b926edc986a870040"
format_stamp: "Formatted at 2026-08-11 01:49:55 on dist-test-slave-9gcw"
server_key: "76660cc0301a69631d8c72dcf7e113c1"
server_key_iv: "990a1ece02da707d12b941580cc4e90c"
server_key_version: "encryptionkey@0"
I20260811 01:49:55.531798 13931 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 01:49:55.535944 13949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:55.537616 13931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:49:55.537830 13931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "ac5e033f0ccf462b926edc986a870040"
format_stamp: "Formatted at 2026-08-11 01:49:55 on dist-test-slave-9gcw"
server_key: "76660cc0301a69631d8c72dcf7e113c1"
server_key_iv: "990a1ece02da707d12b941580cc4e90c"
server_key_version: "encryptionkey@0"
I20260811 01:49:55.538048 13931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:55.576905 13931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:55.581772 13931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:55.582111 13931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:55.603317 13931 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35739
I20260811 01:49:55.603349 14008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35739 every 8 connection(s)
I20260811 01:49:55.604796 13931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:49:55.605556 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 13931
I20260811 01:49:55.605765 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:49:55.606381 22361 external_mini_cluster.cc:1428] Setting key 5c4c26ea1a30434937a658f6ddcb39eb
I20260811 01:49:55.609717 14009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 01:49:55 dist-test-slave-9gcw krb5kdc[13912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 01:49:55.623735 14009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040: Bootstrap starting.
I20260811 01:49:55.628376 14009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:55.629709 14009 log.cc:824] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:55.633652 14012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:55.608433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46233 (local address 127.0.0.1:35739)
0811 01:49:55.609312 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:55.609329 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:55.609385 (+ 56us) server_negotiation.cc:407] Connection header received
0811 01:49:55.611812 (+ 2427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:55.611844 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:55.612404 (+ 560us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:55.612904 (+ 500us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:55.614514 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:55.616763 (+ 2249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:55.618273 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:55.619369 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:55.619587 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:55.623997 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:55.624009 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:55.624031 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:55.624090 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:55.628290 (+ 4200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:55.629119 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:55.629128 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:55.629138 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:55.629424 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:55.629847 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:55.629853 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:55.629859 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:55.630688 (+ 829us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:55.630934 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:55.631679 (+ 745us) server_negotiation.cc:299] Negotiation successful
0811 01:49:55.632197 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":338,"threads_started":1}
I20260811 01:49:55.633908 14009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040: No bootstrap required, opened a new log
I20260811 01:49:55.638365 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:49:55.642987 14009 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac5e033f0ccf462b926edc986a870040" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35739 } }
I20260811 01:49:55.643685 14009 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:55.643849 14009 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac5e033f0ccf462b926edc986a870040, State: Initialized, Role: FOLLOWER
I20260811 01:49:55.644582 14009 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac5e033f0ccf462b926edc986a870040" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35739 } }
I20260811 01:49:55.644853 14009 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:49:55.645009 14009 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:49:55.645251 14009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:55.647706 14009 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac5e033f0ccf462b926edc986a870040" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35739 } }
I20260811 01:49:55.648274 14009 leader_election.cc:302] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac5e033f0ccf462b926edc986a870040; no voters:
I20260811 01:49:55.649021 14009 leader_election.cc:288] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:49:55.649165 14020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:49:55.649876 14020 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040 [term 1 LEADER]: Becoming Leader. State: Replica: ac5e033f0ccf462b926edc986a870040, State: Running, Role: LEADER
I20260811 01:49:55.650686 14020 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac5e033f0ccf462b926edc986a870040" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35739 } }
I20260811 01:49:55.652014 14009 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:49:55.654084 14022 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac5e033f0ccf462b926edc986a870040. Latest consensus state: current_term: 1 leader_uuid: "ac5e033f0ccf462b926edc986a870040" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac5e033f0ccf462b926edc986a870040" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35739 } } }
I20260811 01:49:55.655030 14022 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040 [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:55.655692 14021 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac5e033f0ccf462b926edc986a870040" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac5e033f0ccf462b926edc986a870040" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35739 } } }
I20260811 01:49:55.656778 14021 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040 [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:55.662392 14012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:55.643314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37551 (local address 127.0.0.1:35739)
0811 01:49:55.643521 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:55.643529 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:55.643895 (+ 366us) server_negotiation.cc:407] Connection header received
0811 01:49:55.644117 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:55.644121 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:55.644191 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:55.644315 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:55.646104 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:55.646893 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:55.648221 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:55.648801 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:55.648953 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:55.651970 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:55.651981 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:55.651991 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:49:55.652038 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:55.657766 (+ 5728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:55.659074 (+ 1308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:55.659083 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:55.659090 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:55.659525 (+ 435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:49:55.660096 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:49:55.660105 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:49:55.660110 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:49:55.660661 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:55.661026 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:55.661480 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 01:49:55.661819 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 01:49:55.665397 14025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:49:55.671517 14025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:49:55.680652 14025 catalog_manager.cc:1347] Generated new cluster ID: 90a738950d2e49f4bcf327ef138460f3
I20260811 01:49:55.680791 14025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:49:55.689610 14025 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:49:55.691006 14025 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:49:55.701951 14025 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac5e033f0ccf462b926edc986a870040: Generated new TSK 0
I20260811 01:49:55.702924 14025 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:49:55.710748 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 13931
I20260811 01:49:55.711951 14003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:49:55.805842 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 13931
2026-08-11T01:49:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:49:55 dist-test-slave-9gcw krb5kdc[14042](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:49:55 dist-test-slave-9gcw krb5kdc[14042](info): set up 2 sockets
Aug 11 01:49:55 dist-test-slave-9gcw krb5kdc[14042](info): commencing operation
krb5kdc: starting...
W20260811 01:49:57.033874 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.001s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:49:57 dist-test-slave-9gcw krb5kdc[14042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:49:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:49:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 01:49:57.392856 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38361
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:49:57.760795 14058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:49:57.761124 14058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:49:57.761291 14058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:49:57.768793 14058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:49:57.768922 14058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:49:57.768978 14058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:49:57.769042 14058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:49:57.769093 14058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:49:57.779596 14058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:49:57.780994 14058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:49:57.782457 14058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:49:57.795234 14067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:57.795342 14064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:49:57.795305 14065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:49:57.796309 14058 server_base.cc:1034] running on GCE node
I20260811 01:49:57.797259 14058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:49:57.798790 14058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:49:57.800020 14058 hybrid_clock.cc:648] HybridClock initialized: now 1786412997799968 us; error 48 us; skew 500 ppm
I20260811 01:49:57.800406 14058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:49:57 dist-test-slave-9gcw krb5kdc[14042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412997, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:49:57.803835 14058 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 01:49:57.805547 14058 webserver.cc:460] Webserver started at http://127.0.0.1:35097/ using document root <none> and password file <none>
I20260811 01:49:57.806298 14058 fs_manager.cc:359] Metadata directory not provided
I20260811 01:49:57.806450 14058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:49:57.806775 14058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:49:57.810122 14058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "a32b246a2f7c4ac2b83ad60a7fb8361e"
format_stamp: "Formatted at 2026-08-11 01:49:57 on dist-test-slave-9gcw"
server_key: "119aa19ad3025c47ad7fc678ef91891d"
server_key_iv: "4b46c10995f83a0d5ca4f9c59039c1e2"
server_key_version: "encryptionkey@0"
I20260811 01:49:57.810983 14058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "a32b246a2f7c4ac2b83ad60a7fb8361e"
format_stamp: "Formatted at 2026-08-11 01:49:57 on dist-test-slave-9gcw"
server_key: "119aa19ad3025c47ad7fc678ef91891d"
server_key_iv: "4b46c10995f83a0d5ca4f9c59039c1e2"
server_key_version: "encryptionkey@0"
I20260811 01:49:57.816829 14058 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 01:49:57.820745 14076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:49:57.822232 14058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:49:57.822433 14058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "a32b246a2f7c4ac2b83ad60a7fb8361e"
format_stamp: "Formatted at 2026-08-11 01:49:57 on dist-test-slave-9gcw"
server_key: "119aa19ad3025c47ad7fc678ef91891d"
server_key_iv: "4b46c10995f83a0d5ca4f9c59039c1e2"
server_key_version: "encryptionkey@0"
I20260811 01:49:57.822690 14058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:49:57.845350 14058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:49:57.849730 14058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:49:57.850044 14058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:49:57.870363 14058 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38361
I20260811 01:49:57.870398 14133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38361 every 8 connection(s)
I20260811 01:49:57.871826 14058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:49:57.876166 14134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:49:57.878937 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 14058
I20260811 01:49:57.879164 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:49:57.879729 22361 external_mini_cluster.cc:1428] Setting key 3bb08bb0f928766d8755ec52c5bba337
I20260811 01:49:57.889183 14134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e: Bootstrap starting.
I20260811 01:49:57.892833 14134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e: Neither blocks nor log segments found. Creating new log.
I20260811 01:49:57.894095 14134 log.cc:824] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e: Log is configured to *not* fsync() on all Append() calls
I20260811 01:49:57.894135 14137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:57.882033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51175 (local address 127.0.0.1:38361)
0811 01:49:57.882936 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:57.882949 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:57.882996 (+ 47us) server_negotiation.cc:407] Connection header received
0811 01:49:57.884779 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:57.884805 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:57.885346 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:57.885777 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:57.886397 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:57.887842 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:57.889253 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:57.890014 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:57.890296 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:57.891523 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:57.891531 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:57.891547 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:57.892139 (+ 592us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:57.892239 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:57.892267 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:57.892394 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:57.892633 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 01:49:57.892960 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":367,"threads_started":1}
I20260811 01:49:57.897632 14134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e: No bootstrap required, opened a new log
I20260811 01:49:57.898399 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:49:57.906527 14134 raft_consensus.cc:348] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a32b246a2f7c4ac2b83ad60a7fb8361e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38361 } }
I20260811 01:49:57.906973 14134 raft_consensus.cc:374] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:49:57.907096 14134 raft_consensus.cc:729] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a32b246a2f7c4ac2b83ad60a7fb8361e, State: Initialized, Role: FOLLOWER
I20260811 01:49:57.907891 14134 consensus_queue.cc:260] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a32b246a2f7c4ac2b83ad60a7fb8361e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38361 } }
I20260811 01:49:57.908228 14134 raft_consensus.cc:388] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:49:57.908389 14134 raft_consensus.cc:482] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:49:57.908571 14134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:49:57.910212 14137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:49:57.903039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54525 (local address 127.0.0.1:38361)
0811 01:49:57.903237 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 01:49:57.903245 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:49:57.903658 (+ 413us) server_negotiation.cc:407] Connection header received
0811 01:49:57.903888 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:49:57.903893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:49:57.903963 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:49:57.904119 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:49:57.904732 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:57.905661 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:57.907204 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:49:57.907880 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:49:57.908028 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:49:57.909353 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:49:57.909359 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:49:57.909364 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:49:57.909394 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:49:57.909454 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:49:57.909477 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:49:57.909579 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 01:49:57.909682 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 01:49:57.909797 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 01:49:57.911739 14134 raft_consensus.cc:504] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a32b246a2f7c4ac2b83ad60a7fb8361e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38361 } }
I20260811 01:49:57.912299 14134 leader_election.cc:302] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a32b246a2f7c4ac2b83ad60a7fb8361e; no voters:
I20260811 01:49:57.912935 14134 leader_election.cc:288] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:49:57.913086 14149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:49:57.913944 14149 raft_consensus.cc:686] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e [term 1 LEADER]: Becoming Leader. State: Replica: a32b246a2f7c4ac2b83ad60a7fb8361e, State: Running, Role: LEADER
I20260811 01:49:57.914570 14149 consensus_queue.cc:237] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a32b246a2f7c4ac2b83ad60a7fb8361e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38361 } }
I20260811 01:49:57.915371 14134 sys_catalog.cc:564] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:49:57.917495 14150 sys_catalog.cc:455] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a32b246a2f7c4ac2b83ad60a7fb8361e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a32b246a2f7c4ac2b83ad60a7fb8361e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38361 } } }
I20260811 01:49:57.917908 14150 sys_catalog.cc:458] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:57.918938 14151 sys_catalog.cc:455] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a32b246a2f7c4ac2b83ad60a7fb8361e. Latest consensus state: current_term: 1 leader_uuid: "a32b246a2f7c4ac2b83ad60a7fb8361e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a32b246a2f7c4ac2b83ad60a7fb8361e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38361 } } }
I20260811 01:49:57.919427 14151 sys_catalog.cc:458] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e [sys.catalog]: This master's current role is: LEADER
I20260811 01:49:57.919951 14155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:49:57.927065 14155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:49:57.937750 14155 catalog_manager.cc:1347] Generated new cluster ID: 2543534824194282bb0babdd06f1c5b5
I20260811 01:49:57.937888 14155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:49:57.999446 14155 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:49:58.001526 14155 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:49:58.017928 14155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a32b246a2f7c4ac2b83ad60a7fb8361e: Generated new TSK 0
I20260811 01:49:58.019107 14155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:49:58.030001 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 14058
W20260811 01:49:58.032320 14129 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51175
W20260811 01:49:58.033172 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 14058: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:49:58.033337 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 14058
2026-08-11T01:49:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2229 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:49:58 dist-test-slave-9gcw krb5kdc[14172](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:49:58 dist-test-slave-9gcw krb5kdc[14172](info): set up 2 sockets
Aug 11 01:49:58 dist-test-slave-9gcw krb5kdc[14172](info): commencing operation
krb5kdc: starting...
W20260811 01:49:59.322407 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:49:59 dist-test-slave-9gcw krb5kdc[14172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786412999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:49:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:49:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 01:49:59.673547 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35405
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:50:00.042346 14188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:00.042699 14188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:00.042896 14188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:00.051306 14188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:00.051450 14188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:00.051559 14188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:00.051661 14188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:00.051740 14188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:00.064193 14188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:00.065856 14188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:00.067239 14188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:00.080648 14194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:00.080909 14196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:00.080945 14198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:00.082950 14188 server_base.cc:1034] running on GCE node
I20260811 01:50:00.083874 14188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:00.085556 14188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:00.086814 14188 hybrid_clock.cc:648] HybridClock initialized: now 1786413000086785 us; error 82 us; skew 500 ppm
I20260811 01:50:00.087205 14188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:00 dist-test-slave-9gcw krb5kdc[14172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413000, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:00.090694 14188 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 01:50:00.092379 14188 webserver.cc:460] Webserver started at http://127.0.0.1:45873/ using document root <none> and password file <none>
I20260811 01:50:00.093118 14188 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:00.093338 14188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:00.093675 14188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:00.097267 14188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "aa36fe266fc6420899d4e2349146d83b"
format_stamp: "Formatted at 2026-08-11 01:50:00 on dist-test-slave-9gcw"
server_key: "91a7ecf4abbe236b54d2c08fd0228942"
server_key_iv: "17c6c0c1522a495d09b8dcc3f7a28100"
server_key_version: "encryptionkey@0"
I20260811 01:50:00.098217 14188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "aa36fe266fc6420899d4e2349146d83b"
format_stamp: "Formatted at 2026-08-11 01:50:00 on dist-test-slave-9gcw"
server_key: "91a7ecf4abbe236b54d2c08fd0228942"
server_key_iv: "17c6c0c1522a495d09b8dcc3f7a28100"
server_key_version: "encryptionkey@0"
I20260811 01:50:00.104256 14188 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 01:50:00.108495 14206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:00.110351 14188 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 01:50:00.110605 14188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "aa36fe266fc6420899d4e2349146d83b"
format_stamp: "Formatted at 2026-08-11 01:50:00 on dist-test-slave-9gcw"
server_key: "91a7ecf4abbe236b54d2c08fd0228942"
server_key_iv: "17c6c0c1522a495d09b8dcc3f7a28100"
server_key_version: "encryptionkey@0"
I20260811 01:50:00.110867 14188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:00.117508 14188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:00.122566 14188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:00.122932 14188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:00.143993 14188 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35405
I20260811 01:50:00.144033 14265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35405 every 8 connection(s)
I20260811 01:50:00.145766 14188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:50:00.149874 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 14188
I20260811 01:50:00.150097 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:50:00.150391 14266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:50:00.150630 22361 external_mini_cluster.cc:1428] Setting key bb8dc6de819409417ef8eaa5fa08a368
I20260811 01:50:00.159485 14269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:00.152556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49043 (local address 127.0.0.1:35405)
0811 01:50:00.153514 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:00.153527 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:00.153577 (+ 50us) server_negotiation.cc:407] Connection header received
0811 01:50:00.155648 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:00.155677 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:00.156090 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:00.156584 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:00.157777 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:00.157783 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:00.157796 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:50:00.157814 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:00.157880 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:50:00.157901 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:00.158175 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:00.158321 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 01:50:00.158509 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":389,"threads_started":1}
I20260811 01:50:00.164075 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:50:00.165625 14266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b: Bootstrap starting.
I20260811 01:50:00.169090 14266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:00.170879 14266 log.cc:824] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b: Log is configured to *not* fsync() on all Append() calls
I20260811 01:50:00.172435 14269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:00.169124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42781 (local address 127.0.0.1:35405)
0811 01:50:00.169434 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:00.169441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:00.169884 (+ 443us) server_negotiation.cc:407] Connection header received
0811 01:50:00.170117 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:00.170122 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:00.170197 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:00.170290 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:00.171535 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:00.171545 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:00.171554 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:50:00.171577 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:00.171640 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:50:00.171674 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:00.171805 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:00.171975 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 01:50:00.172033 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260811 01:50:00.174804 14266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b: No bootstrap required, opened a new log
I20260811 01:50:00.185055 14266 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa36fe266fc6420899d4e2349146d83b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35405 } }
I20260811 01:50:00.185614 14266 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:50:00.185715 14266 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa36fe266fc6420899d4e2349146d83b, State: Initialized, Role: FOLLOWER
I20260811 01:50:00.186347 14266 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa36fe266fc6420899d4e2349146d83b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35405 } }
I20260811 01:50:00.186636 14266 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:50:00.186770 14266 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:50:00.186928 14266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:50:00.189965 14266 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa36fe266fc6420899d4e2349146d83b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35405 } }
I20260811 01:50:00.190544 14266 leader_election.cc:302] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa36fe266fc6420899d4e2349146d83b; no voters:
I20260811 01:50:00.191264 14266 leader_election.cc:288] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:50:00.191437 14280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:50:00.192919 14280 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b [term 1 LEADER]: Becoming Leader. State: Replica: aa36fe266fc6420899d4e2349146d83b, State: Running, Role: LEADER
I20260811 01:50:00.193583 14280 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa36fe266fc6420899d4e2349146d83b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35405 } }
I20260811 01:50:00.194139 14266 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:50:00.197396 14281 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa36fe266fc6420899d4e2349146d83b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa36fe266fc6420899d4e2349146d83b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35405 } } }
I20260811 01:50:00.198032 14281 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:00.197420 14282 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa36fe266fc6420899d4e2349146d83b. Latest consensus state: current_term: 1 leader_uuid: "aa36fe266fc6420899d4e2349146d83b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa36fe266fc6420899d4e2349146d83b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35405 } } }
I20260811 01:50:00.198670 14282 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:00.200265 14292 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:50:00.205231 14292 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:50:00.216473 14292 catalog_manager.cc:1347] Generated new cluster ID: e9769f4e43984b7b987eb46ebf2282fd
I20260811 01:50:00.216617 14292 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:50:00.230793 14292 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:50:00.231269 14292 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:50:00.241345 14292 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa36fe266fc6420899d4e2349146d83b: Generated new TSK 0
I20260811 01:50:00.242338 14292 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:50:00.257282 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 14188
W20260811 01:50:00.258543 14261 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49043
W20260811 01:50:00.259343 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 14188: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:50:00.259495 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 14188
2026-08-11T01:50:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2224 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:50:00 dist-test-slave-9gcw krb5kdc[14304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:50:00 dist-test-slave-9gcw krb5kdc[14304](info): set up 2 sockets
Aug 11 01:50:00 dist-test-slave-9gcw krb5kdc[14304](info): commencing operation
krb5kdc: starting...
W20260811 01:50:01.511121 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:50:01 dist-test-slave-9gcw krb5kdc[14304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:50:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:50:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 01:50:01.878362 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37127
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:50:02.255776 14320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:02.256089 14320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:02.256186 14320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:02.263638 14320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:02.263789 14320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:02.263839 14320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:02.263892 14320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:02.263931 14320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:02.275066 14320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:02.276613 14320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:02.278047 14320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:02.291339 14328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:02.291371 14327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:02.291994 14330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:02.292414 14320 server_base.cc:1034] running on GCE node
I20260811 01:50:02.293418 14320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:02.295027 14320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:02.296250 14320 hybrid_clock.cc:648] HybridClock initialized: now 1786413002296197 us; error 46 us; skew 500 ppm
I20260811 01:50:02.296633 14320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:02 dist-test-slave-9gcw krb5kdc[14304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413002, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:02.304975 14320 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 01:50:02.306782 14320 webserver.cc:460] Webserver started at http://127.0.0.1:45763/ using document root <none> and password file <none>
I20260811 01:50:02.307545 14320 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:02.307723 14320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:02.308068 14320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:02.311511 14320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "6cb71ce388be42339ee8c87d96e5301e"
format_stamp: "Formatted at 2026-08-11 01:50:02 on dist-test-slave-9gcw"
server_key: "076bc56e69c367de68ba633d9b32cbe2"
server_key_iv: "9cb487a28f418b707914bc4785eeebfc"
server_key_version: "encryptionkey@0"
I20260811 01:50:02.312398 14320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "6cb71ce388be42339ee8c87d96e5301e"
format_stamp: "Formatted at 2026-08-11 01:50:02 on dist-test-slave-9gcw"
server_key: "076bc56e69c367de68ba633d9b32cbe2"
server_key_iv: "9cb487a28f418b707914bc4785eeebfc"
server_key_version: "encryptionkey@0"
I20260811 01:50:02.318526 14320 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 01:50:02.322538 14338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:02.324098 14320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 01:50:02.324326 14320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "6cb71ce388be42339ee8c87d96e5301e"
format_stamp: "Formatted at 2026-08-11 01:50:02 on dist-test-slave-9gcw"
server_key: "076bc56e69c367de68ba633d9b32cbe2"
server_key_iv: "9cb487a28f418b707914bc4785eeebfc"
server_key_version: "encryptionkey@0"
I20260811 01:50:02.324559 14320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:02.397123 14320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:02.401938 14320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:02.402276 14320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:02.423182 14320 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37127
I20260811 01:50:02.423223 14397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37127 every 8 connection(s)
I20260811 01:50:02.424798 14320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:50:02.426553 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 14320
I20260811 01:50:02.426761 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:50:02.427390 22361 external_mini_cluster.cc:1428] Setting key 2d41ef4443e94df442904917b118e1c8
I20260811 01:50:02.430284 14398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 01:50:02 dist-test-slave-9gcw krb5kdc[14304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 01:50:02.445195 14398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e: Bootstrap starting.
I20260811 01:50:02.450142 14398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:02.451615 14398 log.cc:824] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e: Log is configured to *not* fsync() on all Append() calls
I20260811 01:50:02.455880 14401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:02.429723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:58763 (local address 127.0.0.1:37127)
0811 01:50:02.430571 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:02.430588 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:02.430633 (+ 45us) server_negotiation.cc:407] Connection header received
0811 01:50:02.432378 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:02.432403 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:02.432892 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:02.433360 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:02.435245 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:02.436682 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:02.438787 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:02.439623 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:02.439779 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:02.445266 (+ 5487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:02.445282 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:02.445307 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:02.445376 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:02.449816 (+ 4440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:02.451059 (+ 1243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:02.451069 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:02.451079 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:02.451347 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:02.452014 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:02.452021 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:02.452024 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:02.452789 (+ 765us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:02.453080 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:02.453887 (+ 807us) server_negotiation.cc:299] Negotiation successful
0811 01:50:02.454363 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":380,"threads_started":1}
I20260811 01:50:02.456025 14398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e: No bootstrap required, opened a new log
I20260811 01:50:02.460553 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:50:02.464998 14398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb71ce388be42339ee8c87d96e5301e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37127 } }
I20260811 01:50:02.465780 14398 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:50:02.465937 14398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cb71ce388be42339ee8c87d96e5301e, State: Initialized, Role: FOLLOWER
I20260811 01:50:02.466655 14398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb71ce388be42339ee8c87d96e5301e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37127 } }
I20260811 01:50:02.466910 14398 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:50:02.467057 14398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:50:02.467243 14398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:50:02.469715 14398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb71ce388be42339ee8c87d96e5301e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37127 } }
I20260811 01:50:02.470265 14398 leader_election.cc:302] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cb71ce388be42339ee8c87d96e5301e; no voters:
I20260811 01:50:02.471022 14398 leader_election.cc:288] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:50:02.471125 14414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:50:02.473446 14414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e [term 1 LEADER]: Becoming Leader. State: Replica: 6cb71ce388be42339ee8c87d96e5301e, State: Running, Role: LEADER
I20260811 01:50:02.474099 14398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:50:02.474102 14414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb71ce388be42339ee8c87d96e5301e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37127 } }
I20260811 01:50:02.477382 14417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cb71ce388be42339ee8c87d96e5301e. Latest consensus state: current_term: 1 leader_uuid: "6cb71ce388be42339ee8c87d96e5301e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb71ce388be42339ee8c87d96e5301e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37127 } } }
I20260811 01:50:02.477823 14417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:02.478626 14415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cb71ce388be42339ee8c87d96e5301e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb71ce388be42339ee8c87d96e5301e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37127 } } }
I20260811 01:50:02.479069 14415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:02.485769 14401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:02.465543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:32941 (local address 127.0.0.1:37127)
0811 01:50:02.465721 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:02.465726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:02.466278 (+ 552us) server_negotiation.cc:407] Connection header received
0811 01:50:02.466486 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:02.466492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:02.466563 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:02.466685 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:02.468321 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:02.469110 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:02.470495 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:02.471009 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:02.471157 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:02.474485 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:02.474492 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:02.474497 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:02.474541 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:02.479794 (+ 5253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:02.481390 (+ 1596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:02.481399 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:02.481405 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:02.481661 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:02.482468 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:02.482477 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:02.482483 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:02.483001 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:02.484696 (+ 1695us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:02.484891 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 01:50:02.485131 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 01:50:02.487857 14423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:50:02.493085 14423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:50:02.502986 14423 catalog_manager.cc:1347] Generated new cluster ID: d5b1308bd2df4a1782124d45bce768c4
I20260811 01:50:02.503129 14423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:50:02.511942 14423 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:50:02.513494 14423 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:50:02.520541 14423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cb71ce388be42339ee8c87d96e5301e: Generated new TSK 0
I20260811 01:50:02.521502 14423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:50:02.531416 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 14320
I20260811 01:50:02.533308 14392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:50:02.626318 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 14320
W20260811 01:50:02.651994 14302 connection.cc:535] client connection to 127.0.0.1:37127 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37127): Connection reset by peer (error 104)
2026-08-11T01:50:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2368 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:50:02 dist-test-slave-9gcw krb5kdc[14442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:50:02 dist-test-slave-9gcw krb5kdc[14442](info): set up 2 sockets
Aug 11 01:50:02 dist-test-slave-9gcw krb5kdc[14442](info): commencing operation
krb5kdc: starting...
W20260811 01:50:03.890638 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.004s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:50:04 dist-test-slave-9gcw krb5kdc[14442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:50:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:50:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 01:50:04.262089 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46759
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:50:04.642300 14461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:04.642678 14461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:04.642828 14461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:04.651224 14461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:04.651392 14461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:04.651494 14461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:04.651741 14461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:04.651826 14461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:04.663097 14461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:04.664659 14461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:04.666267 14461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:04.680078 14470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:04.680125 14468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:04.680181 14467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:04.686440 14461 server_base.cc:1034] running on GCE node
I20260811 01:50:04.687364 14461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:04.689239 14461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:04.690502 14461 hybrid_clock.cc:648] HybridClock initialized: now 1786413004690437 us; error 68 us; skew 500 ppm
I20260811 01:50:04.690915 14461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:04 dist-test-slave-9gcw krb5kdc[14442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413004, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:04.694655 14461 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 01:50:04.696363 14461 webserver.cc:460] Webserver started at http://127.0.0.1:40645/ using document root <none> and password file <none>
I20260811 01:50:04.697250 14461 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:04.697378 14461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:04.697677 14461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:04.700953 14461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "9d05a4d801fe49d085b15acaac68d3b6"
format_stamp: "Formatted at 2026-08-11 01:50:04 on dist-test-slave-9gcw"
server_key: "0dfd39582aa936281b6f98d3c9b9ecdd"
server_key_iv: "c90da11b3f537a99646112e6ca6fb125"
server_key_version: "encryptionkey@0"
I20260811 01:50:04.701856 14461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "9d05a4d801fe49d085b15acaac68d3b6"
format_stamp: "Formatted at 2026-08-11 01:50:04 on dist-test-slave-9gcw"
server_key: "0dfd39582aa936281b6f98d3c9b9ecdd"
server_key_iv: "c90da11b3f537a99646112e6ca6fb125"
server_key_version: "encryptionkey@0"
I20260811 01:50:04.707793 14461 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 01:50:04.711839 14479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:04.713618 14461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 01:50:04.713828 14461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "9d05a4d801fe49d085b15acaac68d3b6"
format_stamp: "Formatted at 2026-08-11 01:50:04 on dist-test-slave-9gcw"
server_key: "0dfd39582aa936281b6f98d3c9b9ecdd"
server_key_iv: "c90da11b3f537a99646112e6ca6fb125"
server_key_version: "encryptionkey@0"
I20260811 01:50:04.714026 14461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:04.733130 14461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:04.738332 14461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:04.738687 14461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:04.760243 14461 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46759
I20260811 01:50:04.760273 14551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46759 every 8 connection(s)
I20260811 01:50:04.761847 14461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:50:04.765997 14552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:50:04.769624 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 14461
I20260811 01:50:04.769870 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:50:04.770367 22361 external_mini_cluster.cc:1428] Setting key 27d7137200831c023145b2f9e393c6f7
I20260811 01:50:04.779475 14552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6: Bootstrap starting.
I20260811 01:50:04.783174 14552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:04.784422 14552 log.cc:824] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6: Log is configured to *not* fsync() on all Append() calls
I20260811 01:50:04.786278 14557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:04.772524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:51629 (local address 127.0.0.1:46759)
0811 01:50:04.773567 (+ 1043us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:04.773581 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:04.773743 (+ 162us) server_negotiation.cc:407] Connection header received
0811 01:50:04.775634 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:04.775661 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:04.776168 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:04.776701 (+ 533us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:04.777397 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:04.778885 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:04.780457 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:04.781361 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:04.781625 (+ 264us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:04.783146 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:04.783155 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:04.783176 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:50:04.783902 (+ 726us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:04.783988 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:50:04.784024 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:04.784226 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:04.784550 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 01:50:04.784839 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":478,"threads_started":1}
I20260811 01:50:04.788162 14552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6: No bootstrap required, opened a new log
I20260811 01:50:04.791452 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:50:04.797124 14552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d05a4d801fe49d085b15acaac68d3b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46759 } }
I20260811 01:50:04.797729 14552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:50:04.797899 14552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d05a4d801fe49d085b15acaac68d3b6, State: Initialized, Role: FOLLOWER
I20260811 01:50:04.798575 14552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d05a4d801fe49d085b15acaac68d3b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46759 } }
I20260811 01:50:04.798853 14552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:50:04.799016 14552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:50:04.799216 14552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:50:04.801702 14552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d05a4d801fe49d085b15acaac68d3b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46759 } }
I20260811 01:50:04.802297 14552 leader_election.cc:302] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d05a4d801fe49d085b15acaac68d3b6; no voters:
I20260811 01:50:04.803092 14552 leader_election.cc:288] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:50:04.803221 14567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:50:04.804271 14557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:04.796781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:58573 (local address 127.0.0.1:46759)
0811 01:50:04.796996 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:04.797005 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:04.797523 (+ 518us) server_negotiation.cc:407] Connection header received
0811 01:50:04.797775 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:04.797779 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:04.797849 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:04.797972 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:04.798534 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:04.799566 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:04.800944 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:04.801486 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:04.801623 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:04.803051 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:04.803056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:04.803060 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:50:04.803085 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:04.803126 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:50:04.803142 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:04.803288 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:04.803839 (+ 551us) server_negotiation.cc:299] Negotiation successful
0811 01:50:04.803910 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 01:50:04.806049 14552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:50:04.807426 14567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6 [term 1 LEADER]: Becoming Leader. State: Replica: 9d05a4d801fe49d085b15acaac68d3b6, State: Running, Role: LEADER
I20260811 01:50:04.808269 14567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d05a4d801fe49d085b15acaac68d3b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46759 } }
I20260811 01:50:04.811991 14568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d05a4d801fe49d085b15acaac68d3b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d05a4d801fe49d085b15acaac68d3b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46759 } } }
I20260811 01:50:04.812502 14568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:04.813894 14569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d05a4d801fe49d085b15acaac68d3b6. Latest consensus state: current_term: 1 leader_uuid: "9d05a4d801fe49d085b15acaac68d3b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d05a4d801fe49d085b15acaac68d3b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46759 } } }
I20260811 01:50:04.814208 14569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:04.816186 14582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:50:04.820911 14582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:50:04.830551 14582 catalog_manager.cc:1347] Generated new cluster ID: e22a1cedf8774f2d9d61f2566d93393d
I20260811 01:50:04.830699 14582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:50:04.838214 14582 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:50:04.840257 14582 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:50:04.852213 14582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d05a4d801fe49d085b15acaac68d3b6: Generated new TSK 0
I20260811 01:50:04.853451 14582 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:50:04.863279 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 14461
W20260811 01:50:04.865242 14545 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.180:51629
W20260811 01:50:04.866173 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 14461: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:50:04.866345 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 14461
W20260811 01:50:04.889983 14440 connection.cc:535] client connection to 127.0.0.1:46759 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46759): Connection reset by peer (error 104)
2026-08-11T01:50:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2237 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:50:04 dist-test-slave-9gcw krb5kdc[14592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:50:04 dist-test-slave-9gcw krb5kdc[14592](info): set up 2 sockets
Aug 11 01:50:04 dist-test-slave-9gcw krb5kdc[14592](info): commencing operation
krb5kdc: starting...
W20260811 01:50:06.126698 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.001s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:50:06 dist-test-slave-9gcw krb5kdc[14592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:50:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:50:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 01:50:06.488451 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:50:06.855821 14607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:06.856149 14607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:06.856313 14607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:06.864010 14607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:06.864140 14607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:06.864245 14607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:06.864353 14607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:06.864440 14607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:06.875200 14607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:06.876711 14607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:06.878142 14607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:06.892390 14614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:06.892436 14613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:06.892457 14617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:06.893411 14607 server_base.cc:1034] running on GCE node
I20260811 01:50:06.894272 14607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:06.895876 14607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:06.897116 14607 hybrid_clock.cc:648] HybridClock initialized: now 1786413006897042 us; error 61 us; skew 500 ppm
I20260811 01:50:06.897593 14607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:06 dist-test-slave-9gcw krb5kdc[14592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413006, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:06.901508 14607 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 01:50:06.903267 14607 webserver.cc:460] Webserver started at http://127.0.0.1:43755/ using document root <none> and password file <none>
I20260811 01:50:06.904003 14607 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:06.904122 14607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:06.904439 14607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:06.907951 14607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "6d0be8fe76f5421292ed1a55c81807df"
format_stamp: "Formatted at 2026-08-11 01:50:06 on dist-test-slave-9gcw"
server_key: "5d75f674826164a375b0d6d7069403d0"
server_key_iv: "fe40afab53c729b5d497830c4d77b0f4"
server_key_version: "encryptionkey@0"
I20260811 01:50:06.908828 14607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "6d0be8fe76f5421292ed1a55c81807df"
format_stamp: "Formatted at 2026-08-11 01:50:06 on dist-test-slave-9gcw"
server_key: "5d75f674826164a375b0d6d7069403d0"
server_key_iv: "fe40afab53c729b5d497830c4d77b0f4"
server_key_version: "encryptionkey@0"
I20260811 01:50:06.915522 14607 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 01:50:06.920038 14624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:06.921808 14607 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 01:50:06.922025 14607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "6d0be8fe76f5421292ed1a55c81807df"
format_stamp: "Formatted at 2026-08-11 01:50:06 on dist-test-slave-9gcw"
server_key: "5d75f674826164a375b0d6d7069403d0"
server_key_iv: "fe40afab53c729b5d497830c4d77b0f4"
server_key_version: "encryptionkey@0"
I20260811 01:50:06.922257 14607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:06.932377 14607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:06.937631 14607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:06.937942 14607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:06.959113 14607 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43201
I20260811 01:50:06.959167 14686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43201 every 8 connection(s)
I20260811 01:50:06.960788 14607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:50:06.964161 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 14607
I20260811 01:50:06.964365 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:50:06.964942 22361 external_mini_cluster.cc:1428] Setting key 775fdc5ea84b4e895f9afcfd2cbe29fa
I20260811 01:50:06.965742 14687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:50:06.973749 14690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:06.967059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:40011 (local address 127.0.0.1:43201)
0811 01:50:06.967944 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:06.967956 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:06.968001 (+ 45us) server_negotiation.cc:407] Connection header received
0811 01:50:06.969716 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:06.969739 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:06.970142 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:06.970504 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:06.971841 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:06.971848 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:06.971861 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:50:06.971879 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:06.971964 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:50:06.971990 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:06.972173 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:06.972343 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 01:50:06.972580 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":413,"threads_started":1}
I20260811 01:50:06.978427 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:50:06.982787 14687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df: Bootstrap starting.
I20260811 01:50:06.987306 14687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:06.987643 14690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:06.984618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:42267 (local address 127.0.0.1:43201)
0811 01:50:06.984864 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:06.984877 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:06.985355 (+ 478us) server_negotiation.cc:407] Connection header received
0811 01:50:06.985529 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:06.985535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:06.985603 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:06.985726 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:06.986868 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:06.986874 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:06.986879 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:50:06.986900 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:06.986956 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:50:06.986982 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:06.987135 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:06.987267 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 01:50:06.987344 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 01:50:06.988976 14687 log.cc:824] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df: Log is configured to *not* fsync() on all Append() calls
I20260811 01:50:06.992182 14687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df: No bootstrap required, opened a new log
I20260811 01:50:07.000569 14687 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0be8fe76f5421292ed1a55c81807df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43201 } }
I20260811 01:50:07.001121 14687 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:50:07.001277 14687 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0be8fe76f5421292ed1a55c81807df, State: Initialized, Role: FOLLOWER
I20260811 01:50:07.001956 14687 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0be8fe76f5421292ed1a55c81807df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43201 } }
I20260811 01:50:07.002224 14687 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:50:07.002411 14687 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:50:07.002625 14687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:50:07.005189 14687 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0be8fe76f5421292ed1a55c81807df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43201 } }
I20260811 01:50:07.005798 14687 leader_election.cc:302] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d0be8fe76f5421292ed1a55c81807df; no voters:
I20260811 01:50:07.006470 14687 leader_election.cc:288] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:50:07.006656 14698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:50:07.007589 14698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df [term 1 LEADER]: Becoming Leader. State: Replica: 6d0be8fe76f5421292ed1a55c81807df, State: Running, Role: LEADER
I20260811 01:50:07.008253 14698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0be8fe76f5421292ed1a55c81807df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43201 } }
I20260811 01:50:07.008919 14687 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:50:07.012063 14700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d0be8fe76f5421292ed1a55c81807df. Latest consensus state: current_term: 1 leader_uuid: "6d0be8fe76f5421292ed1a55c81807df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0be8fe76f5421292ed1a55c81807df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43201 } } }
I20260811 01:50:07.012635 14700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:07.012744 14699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d0be8fe76f5421292ed1a55c81807df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0be8fe76f5421292ed1a55c81807df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43201 } } }
I20260811 01:50:07.013105 14699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:07.017927 14704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:50:07.023682 14704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:50:07.033396 14704 catalog_manager.cc:1347] Generated new cluster ID: 31619d0029d9478598d231cd16560351
I20260811 01:50:07.033545 14704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:50:07.048776 14704 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:50:07.049430 14704 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:50:07.070776 14704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d0be8fe76f5421292ed1a55c81807df: Generated new TSK 0
I20260811 01:50:07.072082 14704 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:50:07.080669 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 14607
W20260811 01:50:07.081990 14682 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.180:40011
W20260811 01:50:07.082829 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 14607: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:50:07.083005 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 14607
2026-08-11T01:50:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2220 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:50:07 dist-test-slave-9gcw krb5kdc[14723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:50:07 dist-test-slave-9gcw krb5kdc[14723](info): set up 2 sockets
Aug 11 01:50:07 dist-test-slave-9gcw krb5kdc[14723](info): commencing operation
krb5kdc: starting...
W20260811 01:50:08.380497 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:50:08 dist-test-slave-9gcw krb5kdc[14723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:50:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:50:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 01:50:08.735023 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38495
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:50:09.112865 14740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:09.113274 14740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:09.113448 14740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:09.121579 14740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:09.121856 14740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:09.121970 14740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:09.122102 14740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:09.122184 14740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:09.133116 14740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38495
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:09.134675 14740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:09.136149 14740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:09.149760 14746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:09.149677 14749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:09.149729 14747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:09.150720 14740 server_base.cc:1034] running on GCE node
I20260811 01:50:09.151582 14740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:09.153415 14740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:09.154675 14740 hybrid_clock.cc:648] HybridClock initialized: now 1786413009154626 us; error 62 us; skew 500 ppm
I20260811 01:50:09.155181 14740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:09 dist-test-slave-9gcw krb5kdc[14723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413009, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:09.165671 14740 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 01:50:09.167375 14740 webserver.cc:460] Webserver started at http://127.0.0.1:44163/ using document root <none> and password file <none>
I20260811 01:50:09.168076 14740 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:09.168185 14740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:09.168469 14740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:09.172068 14740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "787c296c51c640efbd68210ea246c798"
format_stamp: "Formatted at 2026-08-11 01:50:09 on dist-test-slave-9gcw"
server_key: "7162cb78322651475be0ea48211270ca"
server_key_iv: "c4616759e6db19d5630f1c11c344796e"
server_key_version: "encryptionkey@0"
I20260811 01:50:09.172894 14740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "787c296c51c640efbd68210ea246c798"
format_stamp: "Formatted at 2026-08-11 01:50:09 on dist-test-slave-9gcw"
server_key: "7162cb78322651475be0ea48211270ca"
server_key_iv: "c4616759e6db19d5630f1c11c344796e"
server_key_version: "encryptionkey@0"
I20260811 01:50:09.178798 14740 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 01:50:09.182512 14757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:09.184037 14740 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:50:09.184228 14740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "787c296c51c640efbd68210ea246c798"
format_stamp: "Formatted at 2026-08-11 01:50:09 on dist-test-slave-9gcw"
server_key: "7162cb78322651475be0ea48211270ca"
server_key_iv: "c4616759e6db19d5630f1c11c344796e"
server_key_version: "encryptionkey@0"
I20260811 01:50:09.184479 14740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:09.211858 14740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:09.216719 14740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:09.217057 14740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:09.238416 14740 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38495
I20260811 01:50:09.238509 14817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38495 every 8 connection(s)
I20260811 01:50:09.239892 14740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:50:09.242628 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 14740
I20260811 01:50:09.242833 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:50:09.243287 22361 external_mini_cluster.cc:1428] Setting key 5b48e152180c7b6d71cac0620b385ae0
I20260811 01:50:09.244956 14818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:50:09.262497 14818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798: Bootstrap starting.
Aug 11 01:50:09 dist-test-slave-9gcw krb5kdc[14723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 01:50:09.266559 14818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:09.267887 14818 log.cc:824] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798: Log is configured to *not* fsync() on all Append() calls
I20260811 01:50:09.271258 14818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798: No bootstrap required, opened a new log
I20260811 01:50:09.274361 14824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:09.245409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:50179 (local address 127.0.0.1:38495)
0811 01:50:09.246286 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:09.246305 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:09.246365 (+ 60us) server_negotiation.cc:407] Connection header received
0811 01:50:09.249077 (+ 2712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:09.249122 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:09.249867 (+ 745us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:09.250414 (+ 547us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:09.252816 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:09.255793 (+ 2977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:09.258247 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:09.259372 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:09.259647 (+ 275us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:09.264891 (+ 5244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:09.264900 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:09.264919 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:09.264966 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:09.268721 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:09.269950 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:09.269959 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:09.269966 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:09.270164 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:09.270725 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:09.270729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:09.270731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:09.271352 (+ 621us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:09.271719 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:09.272635 (+ 916us) server_negotiation.cc:299] Negotiation successful
0811 01:50:09.273045 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":330,"threads_started":1}
I20260811 01:50:09.279081 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:50:09.280423 14818 raft_consensus.cc:348] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787c296c51c640efbd68210ea246c798" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38495 } }
I20260811 01:50:09.280854 14818 raft_consensus.cc:374] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:50:09.280961 14818 raft_consensus.cc:729] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787c296c51c640efbd68210ea246c798, State: Initialized, Role: FOLLOWER
I20260811 01:50:09.281638 14818 consensus_queue.cc:260] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787c296c51c640efbd68210ea246c798" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38495 } }
I20260811 01:50:09.281893 14818 raft_consensus.cc:388] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:50:09.282064 14818 raft_consensus.cc:482] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:50:09.282263 14818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:50:09.285401 14818 raft_consensus.cc:504] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787c296c51c640efbd68210ea246c798" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38495 } }
I20260811 01:50:09.285975 14818 leader_election.cc:302] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 787c296c51c640efbd68210ea246c798; no voters:
I20260811 01:50:09.286697 14818 leader_election.cc:288] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:50:09.286783 14832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:50:09.289817 14818 sys_catalog.cc:564] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:50:09.293474 14832 raft_consensus.cc:686] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798 [term 1 LEADER]: Becoming Leader. State: Replica: 787c296c51c640efbd68210ea246c798, State: Running, Role: LEADER
I20260811 01:50:09.294452 14832 consensus_queue.cc:237] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787c296c51c640efbd68210ea246c798" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38495 } }
I20260811 01:50:09.298521 14835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "787c296c51c640efbd68210ea246c798" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787c296c51c640efbd68210ea246c798" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38495 } } }
I20260811 01:50:09.298978 14835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:09.298422 14834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 787c296c51c640efbd68210ea246c798. Latest consensus state: current_term: 1 leader_uuid: "787c296c51c640efbd68210ea246c798" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787c296c51c640efbd68210ea246c798" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38495 } } }
I20260811 01:50:09.299160 14834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:09.301581 14824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:09.284166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:49719 (local address 127.0.0.1:38495)
0811 01:50:09.284368 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:09.284378 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:09.284826 (+ 448us) server_negotiation.cc:407] Connection header received
0811 01:50:09.285027 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:09.285033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:09.285114 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:09.285263 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:09.287034 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:09.288383 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:09.290128 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:09.290611 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:09.290756 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:09.293782 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:09.293789 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:09.293794 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:09.293846 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:09.297250 (+ 3404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:09.298485 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:09.298493 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:09.298502 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:09.298702 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:09.299440 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:09.299445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:09.299448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:09.299873 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:09.300143 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:09.300928 (+ 785us) server_negotiation.cc:299] Negotiation successful
0811 01:50:09.301123 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 01:50:09.305837 14841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:50:09.312507 14841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:50:09.322638 14841 catalog_manager.cc:1347] Generated new cluster ID: 3321936aec1e4b1087ba6913ada91cbf
I20260811 01:50:09.322793 14841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:50:09.343786 14841 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:50:09.345705 14841 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:50:09.354992 14841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 787c296c51c640efbd68210ea246c798: Generated new TSK 0
I20260811 01:50:09.356127 14841 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:50:09.373082 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 14740
I20260811 01:50:09.375124 14811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:50:09.472465 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 14740
W20260811 01:50:09.495384 14721 connection.cc:535] client connection to 127.0.0.1:38495 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38495): Connection reset by peer (error 104)
2026-08-11T01:50:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2388 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:50:09 dist-test-slave-9gcw krb5kdc[14858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:50:09 dist-test-slave-9gcw krb5kdc[14858](info): set up 2 sockets
Aug 11 01:50:09 dist-test-slave-9gcw krb5kdc[14858](info): commencing operation
krb5kdc: starting...
W20260811 01:50:10.713044 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.001s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:50:10 dist-test-slave-9gcw krb5kdc[14858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:50:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:50:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 01:50:11.079452 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45151
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:50:11.457937 14876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:11.458287 14876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:11.458506 14876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:11.466132 14876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:11.466285 14876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:11.466424 14876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:11.466552 14876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:11.466639 14876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:11.477916 14876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45151
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:11.479483 14876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:11.481062 14876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:11.495121 14882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:11.495164 14887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:11.495152 14881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:11.496105 14876 server_base.cc:1034] running on GCE node
I20260811 01:50:11.496935 14876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:11.498656 14876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:11.499902 14876 hybrid_clock.cc:648] HybridClock initialized: now 1786413011499864 us; error 50 us; skew 500 ppm
I20260811 01:50:11.500321 14876 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:11 dist-test-slave-9gcw krb5kdc[14858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413011, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:11.504009 14876 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 01:50:11.505816 14876 webserver.cc:460] Webserver started at http://127.0.0.1:42411/ using document root <none> and password file <none>
I20260811 01:50:11.506541 14876 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:11.506651 14876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:11.506932 14876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:11.510164 14876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "b2cd24067a8e4fffa4405dfbcec1c644"
format_stamp: "Formatted at 2026-08-11 01:50:11 on dist-test-slave-9gcw"
server_key: "0a4495e6d954049136373199268ffeef"
server_key_iv: "f7582aaf64f2d1dc922de6ab96851e80"
server_key_version: "encryptionkey@0"
I20260811 01:50:11.510952 14876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "b2cd24067a8e4fffa4405dfbcec1c644"
format_stamp: "Formatted at 2026-08-11 01:50:11 on dist-test-slave-9gcw"
server_key: "0a4495e6d954049136373199268ffeef"
server_key_iv: "f7582aaf64f2d1dc922de6ab96851e80"
server_key_version: "encryptionkey@0"
I20260811 01:50:11.516530 14876 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 01:50:11.520378 14895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:11.521979 14876 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 01:50:11.522176 14876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "b2cd24067a8e4fffa4405dfbcec1c644"
format_stamp: "Formatted at 2026-08-11 01:50:11 on dist-test-slave-9gcw"
server_key: "0a4495e6d954049136373199268ffeef"
server_key_iv: "f7582aaf64f2d1dc922de6ab96851e80"
server_key_version: "encryptionkey@0"
I20260811 01:50:11.522392 14876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:11.552656 14876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:11.557467 14876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:11.557801 14876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:11.579733 14876 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45151
I20260811 01:50:11.579780 14952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45151 every 8 connection(s)
I20260811 01:50:11.581518 14876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:50:11.586032 14953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:50:11.586192 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 14876
I20260811 01:50:11.586369 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:50:11.586964 22361 external_mini_cluster.cc:1428] Setting key 206ebfccf37e2ebb1c1d1bb30ca5d4c5
I20260811 01:50:11.600906 14953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644: Bootstrap starting.
I20260811 01:50:11.603322 14957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:11.589096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:60357 (local address 127.0.0.1:45151)
0811 01:50:11.590191 (+ 1095us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:11.590205 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:11.590253 (+ 48us) server_negotiation.cc:407] Connection header received
0811 01:50:11.592841 (+ 2588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:11.592870 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:11.593448 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:11.593842 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:11.594504 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:11.596090 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:11.598152 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:11.598970 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:11.599155 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:11.600544 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:11.600552 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:11.600569 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:50:11.601404 (+ 835us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:11.601494 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:50:11.601526 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:11.601667 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:11.601945 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 01:50:11.602175 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":769,"thread_start_us":472,"threads_started":1}
I20260811 01:50:11.604413 14953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:11.605645 14953 log.cc:824] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644: Log is configured to *not* fsync() on all Append() calls
I20260811 01:50:11.607645 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:50:11.609136 14953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644: No bootstrap required, opened a new log
I20260811 01:50:11.620216 14953 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2cd24067a8e4fffa4405dfbcec1c644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45151 } }
I20260811 01:50:11.620729 14957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:11.613376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:58255 (local address 127.0.0.1:45151)
0811 01:50:11.613615 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:11.613622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:11.613845 (+ 223us) server_negotiation.cc:407] Connection header received
0811 01:50:11.614112 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:11.614116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:11.614184 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:11.614309 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:11.614995 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:11.615964 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:11.617959 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:11.618453 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:11.618554 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:11.619858 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:11.619863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:11.619866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:50:11.619885 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:11.619939 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:50:11.619955 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:11.620044 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:11.620292 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 01:50:11.620340 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 01:50:11.620774 14953 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:50:11.621014 14953 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2cd24067a8e4fffa4405dfbcec1c644, State: Initialized, Role: FOLLOWER
I20260811 01:50:11.621747 14953 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2cd24067a8e4fffa4405dfbcec1c644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45151 } }
I20260811 01:50:11.622036 14953 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:50:11.622200 14953 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:50:11.622419 14953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:50:11.624739 14953 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2cd24067a8e4fffa4405dfbcec1c644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45151 } }
I20260811 01:50:11.625460 14953 leader_election.cc:302] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2cd24067a8e4fffa4405dfbcec1c644; no voters:
I20260811 01:50:11.626063 14953 leader_election.cc:288] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:50:11.626202 14964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:50:11.627417 14964 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644 [term 1 LEADER]: Becoming Leader. State: Replica: b2cd24067a8e4fffa4405dfbcec1c644, State: Running, Role: LEADER
I20260811 01:50:11.627965 14964 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2cd24067a8e4fffa4405dfbcec1c644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45151 } }
I20260811 01:50:11.628423 14953 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:50:11.631253 14966 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2cd24067a8e4fffa4405dfbcec1c644. Latest consensus state: current_term: 1 leader_uuid: "b2cd24067a8e4fffa4405dfbcec1c644" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2cd24067a8e4fffa4405dfbcec1c644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45151 } } }
I20260811 01:50:11.631680 14966 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:11.632270 14965 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2cd24067a8e4fffa4405dfbcec1c644" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2cd24067a8e4fffa4405dfbcec1c644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45151 } } }
I20260811 01:50:11.632596 14965 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:11.633560 14971 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:50:11.639659 14971 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:50:11.649650 14971 catalog_manager.cc:1347] Generated new cluster ID: 55f51eac7e7f4d7ab4100a582a0bf2c1
I20260811 01:50:11.649796 14971 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:50:11.674232 14971 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:50:11.676038 14971 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:50:11.685456 14971 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2cd24067a8e4fffa4405dfbcec1c644: Generated new TSK 0
I20260811 01:50:11.686654 14971 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:50:11.693660 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 14876
W20260811 01:50:11.695741 14947 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.180:60357
W20260811 01:50:11.696605 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 14876: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:50:11.696784 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 14876
W20260811 01:50:11.718900 14856 connection.cc:535] client connection to 127.0.0.1:45151 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45151): Connection reset by peer (error 104)
2026-08-11T01:50:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2220 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:50:11 dist-test-slave-9gcw krb5kdc[14989](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:50:11 dist-test-slave-9gcw krb5kdc[14989](info): set up 2 sockets
Aug 11 01:50:11 dist-test-slave-9gcw krb5kdc[14989](info): commencing operation
krb5kdc: starting...
W20260811 01:50:12.964480 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:50:13 dist-test-slave-9gcw krb5kdc[14989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:50:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:50:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 01:50:13.321064 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37171
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:50:13.703351 15004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:13.703675 15004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:13.703809 15004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:13.711340 15004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:13.711472 15004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:13.711570 15004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:13.711673 15004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:13.711787 15004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:13.722174 15004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37171
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:13.723637 15004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:13.725169 15004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:13.738205 15013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:13.738183 15009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:13.738274 15010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:13.739702 15004 server_base.cc:1034] running on GCE node
I20260811 01:50:13.740569 15004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:13.742206 15004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:13.743435 15004 hybrid_clock.cc:648] HybridClock initialized: now 1786413013743390 us; error 37 us; skew 500 ppm
I20260811 01:50:13.743780 15004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:13 dist-test-slave-9gcw krb5kdc[14989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413013, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:13.747360 15004 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 01:50:13.749091 15004 webserver.cc:460] Webserver started at http://127.0.0.1:36769/ using document root <none> and password file <none>
I20260811 01:50:13.749915 15004 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:13.750056 15004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:13.750406 15004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:13.753728 15004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "d73c9e62d90f4c82a204b22e4e73498e"
format_stamp: "Formatted at 2026-08-11 01:50:13 on dist-test-slave-9gcw"
server_key: "cc7e311b99ea4a696f9b0153c35407b4"
server_key_iv: "581c234723ff4513b7beada2ca2e9ffc"
server_key_version: "encryptionkey@0"
I20260811 01:50:13.754557 15004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "d73c9e62d90f4c82a204b22e4e73498e"
format_stamp: "Formatted at 2026-08-11 01:50:13 on dist-test-slave-9gcw"
server_key: "cc7e311b99ea4a696f9b0153c35407b4"
server_key_iv: "581c234723ff4513b7beada2ca2e9ffc"
server_key_version: "encryptionkey@0"
I20260811 01:50:13.760289 15004 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 01:50:13.764163 15020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:13.765874 15004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:50:13.766078 15004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "d73c9e62d90f4c82a204b22e4e73498e"
format_stamp: "Formatted at 2026-08-11 01:50:13 on dist-test-slave-9gcw"
server_key: "cc7e311b99ea4a696f9b0153c35407b4"
server_key_iv: "581c234723ff4513b7beada2ca2e9ffc"
server_key_version: "encryptionkey@0"
I20260811 01:50:13.766300 15004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:13.772710 15004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:13.777472 15004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:13.777806 15004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:13.798424 15004 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37171
I20260811 01:50:13.798473 15080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37171 every 8 connection(s)
I20260811 01:50:13.800004 15004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:50:13.804114 15081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:50:13.806584 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 15004
I20260811 01:50:13.806802 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:50:13.807312 22361 external_mini_cluster.cc:1428] Setting key e6541b31b3c0604345b12b79e97e2d9e
I20260811 01:50:13.815805 15084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:13.809340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:60371 (local address 127.0.0.1:37171)
0811 01:50:13.810328 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:13.810341 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:13.810385 (+ 44us) server_negotiation.cc:407] Connection header received
0811 01:50:13.812129 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:13.812155 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:13.812533 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:13.812915 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:13.814089 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:13.814095 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:13.814109 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:50:13.814127 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:13.814194 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:50:13.814213 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:13.814410 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:13.814517 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 01:50:13.814756 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":770,"thread_start_us":493,"threads_started":1}
I20260811 01:50:13.817054 15081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e: Bootstrap starting.
I20260811 01:50:13.819826 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:50:13.820662 15081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:13.821798 15081 log.cc:824] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e: Log is configured to *not* fsync() on all Append() calls
I20260811 01:50:13.825726 15081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e: No bootstrap required, opened a new log
I20260811 01:50:13.827416 15084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:13.824625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:53577 (local address 127.0.0.1:37171)
0811 01:50:13.824887 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:13.824896 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:13.825336 (+ 440us) server_negotiation.cc:407] Connection header received
0811 01:50:13.825548 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:13.825559 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:13.825609 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:13.825724 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:13.826730 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:13.826742 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:13.826745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:50:13.826767 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:13.826807 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:50:13.826822 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:13.826946 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:13.827080 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 01:50:13.827147 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 01:50:13.835439 15081 raft_consensus.cc:348] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d73c9e62d90f4c82a204b22e4e73498e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37171 } }
I20260811 01:50:13.835955 15081 raft_consensus.cc:374] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:50:13.836079 15081 raft_consensus.cc:729] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d73c9e62d90f4c82a204b22e4e73498e, State: Initialized, Role: FOLLOWER
I20260811 01:50:13.836747 15081 consensus_queue.cc:260] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d73c9e62d90f4c82a204b22e4e73498e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37171 } }
I20260811 01:50:13.837076 15081 raft_consensus.cc:388] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:50:13.837307 15081 raft_consensus.cc:482] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:50:13.837548 15081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:50:13.840204 15081 raft_consensus.cc:504] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d73c9e62d90f4c82a204b22e4e73498e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37171 } }
I20260811 01:50:13.840744 15081 leader_election.cc:302] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d73c9e62d90f4c82a204b22e4e73498e; no voters:
I20260811 01:50:13.841526 15081 leader_election.cc:288] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:50:13.841667 15092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:50:13.843019 15092 raft_consensus.cc:686] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e [term 1 LEADER]: Becoming Leader. State: Replica: d73c9e62d90f4c82a204b22e4e73498e, State: Running, Role: LEADER
I20260811 01:50:13.843565 15092 consensus_queue.cc:237] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d73c9e62d90f4c82a204b22e4e73498e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37171 } }
I20260811 01:50:13.844282 15081 sys_catalog.cc:564] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:50:13.846798 15096 sys_catalog.cc:455] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d73c9e62d90f4c82a204b22e4e73498e. Latest consensus state: current_term: 1 leader_uuid: "d73c9e62d90f4c82a204b22e4e73498e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d73c9e62d90f4c82a204b22e4e73498e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37171 } } }
I20260811 01:50:13.847076 15094 sys_catalog.cc:455] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d73c9e62d90f4c82a204b22e4e73498e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d73c9e62d90f4c82a204b22e4e73498e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37171 } } }
I20260811 01:50:13.847296 15096 sys_catalog.cc:458] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:13.847376 15094 sys_catalog.cc:458] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:13.849769 15102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:50:13.854527 15102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:50:13.863700 15102 catalog_manager.cc:1347] Generated new cluster ID: 82c5b475b8b04879870d6c8035f78420
I20260811 01:50:13.863848 15102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:50:13.882838 15102 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:50:13.883322 15102 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:50:13.890463 15102 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d73c9e62d90f4c82a204b22e4e73498e: Generated new TSK 0
I20260811 01:50:13.891460 15102 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:50:13.907140 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 15004
W20260811 01:50:13.908815 15076 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.180:60371
W20260811 01:50:13.909709 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 15004: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:50:13.909893 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 15004
2026-08-11T01:50:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2216 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:50:14 dist-test-slave-9gcw krb5kdc[15118](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:50:14 dist-test-slave-9gcw krb5kdc[15118](info): set up 2 sockets
Aug 11 01:50:14 dist-test-slave-9gcw krb5kdc[15118](info): commencing operation
krb5kdc: starting...
W20260811 01:50:15.156291 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.002s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:50:15 dist-test-slave-9gcw krb5kdc[15118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:50:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:50:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:50:15.512341 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:40723
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:38187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:40723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:50:15.875166 15135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:15.875522 15135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:15.875747 15135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:15.883176 15135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:15.883373 15135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:15.883499 15135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:15.883621 15135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:15.883704 15135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:15.894177 15135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:38187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:40723
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:40723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:15.895704 15135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:15.897130 15135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:15.909991 15141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:15.910032 15143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:15.909967 15145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:15.910873 15135 server_base.cc:1034] running on GCE node
I20260811 01:50:15.911769 15135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:15.913390 15135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:15.914633 15135 hybrid_clock.cc:648] HybridClock initialized: now 1786413015914592 us; error 54 us; skew 500 ppm
I20260811 01:50:15.915006 15135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:15 dist-test-slave-9gcw krb5kdc[15118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413015, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:15.923604 15135 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:50:15.925413 15135 webserver.cc:460] Webserver started at http://127.21.214.126:40385/ using document root <none> and password file <none>
I20260811 01:50:15.926136 15135 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:15.926299 15135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:15.926657 15135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:15.929852 15135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "c342abcefc6046a49873a436ab06a73b"
format_stamp: "Formatted at 2026-08-11 01:50:15 on dist-test-slave-9gcw"
server_key: "1001d3b4eccb3bf3163de21f937bd419"
server_key_iv: "ec1bdb3598db1ef74c039e5b9ad429d0"
server_key_version: "encryptionkey@0"
I20260811 01:50:15.930927 15135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "c342abcefc6046a49873a436ab06a73b"
format_stamp: "Formatted at 2026-08-11 01:50:15 on dist-test-slave-9gcw"
server_key: "1001d3b4eccb3bf3163de21f937bd419"
server_key_iv: "ec1bdb3598db1ef74c039e5b9ad429d0"
server_key_version: "encryptionkey@0"
I20260811 01:50:15.936600 15135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 01:50:15.940833 15152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:15.942425 15135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 01:50:15.942629 15135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "c342abcefc6046a49873a436ab06a73b"
format_stamp: "Formatted at 2026-08-11 01:50:15 on dist-test-slave-9gcw"
server_key: "1001d3b4eccb3bf3163de21f937bd419"
server_key_iv: "ec1bdb3598db1ef74c039e5b9ad429d0"
server_key_version: "encryptionkey@0"
I20260811 01:50:15.942907 15135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:15.975865 15135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:15.980708 15135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:15.981046 15135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:16.001323 15135 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:40723
I20260811 01:50:16.001368 15207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:40723 every 8 connection(s)
I20260811 01:50:16.002902 15135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:50:16.007325 15208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:50:16.008656 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 15135
I20260811 01:50:16.008878 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:50:16.009558 22361 external_mini_cluster.cc:1428] Setting key 3a2bf99ec6e111d93c17c835b951fe33
I20260811 01:50:16.023803 15208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b: Bootstrap starting.
Aug 11 01:50:16 dist-test-slave-9gcw krb5kdc[15118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:50:16.027894 15208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:16.029049 15208 log.cc:824] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b: Log is configured to *not* fsync() on all Append() calls
I20260811 01:50:16.032269 15208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b: No bootstrap required, opened a new log
I20260811 01:50:16.036114 15215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:16.011545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60861 (local address 127.21.214.126:40723)
0811 01:50:16.012353 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:16.012367 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:16.012416 (+ 49us) server_negotiation.cc:407] Connection header received
0811 01:50:16.014400 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:16.014424 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:16.014836 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:16.015186 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:16.017493 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:16.018822 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:16.020942 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:16.022640 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:16.022788 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:16.027999 (+ 5211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:16.028028 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:16.028050 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:16.028114 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:16.031533 (+ 3419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:16.032415 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:16.032422 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:16.032430 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:16.032599 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:16.033112 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:16.033122 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:16.033124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:16.033748 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:16.034140 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:16.034654 (+ 514us) server_negotiation.cc:299] Negotiation successful
0811 01:50:16.035013 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":384,"threads_started":1}
I20260811 01:50:16.040385 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:50:16.041019 15208 raft_consensus.cc:348] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c342abcefc6046a49873a436ab06a73b" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40723 } }
I20260811 01:50:16.041491 15208 raft_consensus.cc:374] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:50:16.041608 15208 raft_consensus.cc:729] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c342abcefc6046a49873a436ab06a73b, State: Initialized, Role: FOLLOWER
I20260811 01:50:16.042209 15208 consensus_queue.cc:260] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c342abcefc6046a49873a436ab06a73b" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40723 } }
I20260811 01:50:16.042476 15208 raft_consensus.cc:388] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:50:16.042627 15208 raft_consensus.cc:482] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:50:16.042819 15208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:50:16.045187 15208 raft_consensus.cc:504] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c342abcefc6046a49873a436ab06a73b" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40723 } }
I20260811 01:50:16.045851 15208 leader_election.cc:302] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c342abcefc6046a49873a436ab06a73b; no voters:
I20260811 01:50:16.046722 15208 leader_election.cc:288] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:50:16.046838 15223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:50:16.047597 15223 raft_consensus.cc:686] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b [term 1 LEADER]: Becoming Leader. State: Replica: c342abcefc6046a49873a436ab06a73b, State: Running, Role: LEADER
I20260811 01:50:16.048278 15223 consensus_queue.cc:237] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c342abcefc6046a49873a436ab06a73b" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40723 } }
I20260811 01:50:16.051081 15208 sys_catalog.cc:564] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:50:16.056136 15224 sys_catalog.cc:455] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c342abcefc6046a49873a436ab06a73b. Latest consensus state: current_term: 1 leader_uuid: "c342abcefc6046a49873a436ab06a73b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c342abcefc6046a49873a436ab06a73b" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40723 } } }
I20260811 01:50:16.056947 15224 sys_catalog.cc:458] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:16.064088 15225 sys_catalog.cc:455] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c342abcefc6046a49873a436ab06a73b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c342abcefc6046a49873a436ab06a73b" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 40723 } } }
I20260811 01:50:16.064512 15225 sys_catalog.cc:458] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:16.065442 15234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:50:16.070482 15215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:16.045467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54643 (local address 127.21.214.126:40723)
0811 01:50:16.045659 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:16.045666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:16.046205 (+ 539us) server_negotiation.cc:407] Connection header received
0811 01:50:16.046384 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:16.046392 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:16.046490 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:16.046609 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:16.049216 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:16.050715 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:16.053130 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:16.054368 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:16.054557 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:16.059314 (+ 4757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:16.059327 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:16.059336 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:16.059403 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:16.066042 (+ 6639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:16.067316 (+ 1274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:16.067321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:16.067324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:16.067498 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:16.068279 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:16.068284 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:16.068287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:16.068732 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:16.069097 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:16.069857 (+ 760us) server_negotiation.cc:299] Negotiation successful
0811 01:50:16.070075 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 01:50:16.071637 15234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:50:16.082017 15234 catalog_manager.cc:1347] Generated new cluster ID: 0d4a4ac22a664cbd9be7781d1eef342e
I20260811 01:50:16.082158 15234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:50:16.099260 15234 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:50:16.101265 15234 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:50:16.111509 15234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c342abcefc6046a49873a436ab06a73b: Generated new TSK 0
I20260811 01:50:16.113000 15234 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:50:16.127481 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 15135
I20260811 01:50:16.128793 15203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:50:16.224567 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 15135
W20260811 01:50:16.248824 15115 connection.cc:535] client connection to 127.21.214.126:40723 recv error: Network error: failed to read from TLS socket (remote: 127.21.214.126:40723): Connection reset by peer (error 104)
2026-08-11T01:50:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:50:16 dist-test-slave-9gcw krb5kdc[15251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:50:16 dist-test-slave-9gcw krb5kdc[15251](info): set up 2 sockets
Aug 11 01:50:16 dist-test-slave-9gcw krb5kdc[15251](info): commencing operation
krb5kdc: starting...
W20260811 01:50:17.466475 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:50:17 dist-test-slave-9gcw krb5kdc[15251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:50:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:50:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:50:17.829502 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:38755
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:42243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:38755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:50:18.197314 15265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:18.197672 15265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:18.197786 15265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:18.206053 15265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:18.206197 15265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:18.206251 15265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:18.206323 15265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:18.206373 15265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:18.218061 15265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:42243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:38755
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:38755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:18.219578 15265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:18.221047 15265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:18.234800 15271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:18.234851 15272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:18.235191 15274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:18.235962 15265 server_base.cc:1034] running on GCE node
I20260811 01:50:18.236833 15265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:18.238415 15265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:18.239686 15265 hybrid_clock.cc:648] HybridClock initialized: now 1786413018239642 us; error 54 us; skew 500 ppm
I20260811 01:50:18.240120 15265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:18 dist-test-slave-9gcw krb5kdc[15251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413018, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:18.243594 15265 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:50:18.245306 15265 webserver.cc:460] Webserver started at http://127.21.214.126:33263/ using document root <none> and password file <none>
I20260811 01:50:18.246031 15265 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:18.246138 15265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:18.246408 15265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:18.249660 15265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "bdd5a45b7eeb4324a3a728ad0e62b2e5"
format_stamp: "Formatted at 2026-08-11 01:50:18 on dist-test-slave-9gcw"
server_key: "cbcd1a98ad6eabb7890cad975be074fe"
server_key_iv: "fb20a1ecb19e9aa105735f91ce9898bd"
server_key_version: "encryptionkey@0"
I20260811 01:50:18.250430 15265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "bdd5a45b7eeb4324a3a728ad0e62b2e5"
format_stamp: "Formatted at 2026-08-11 01:50:18 on dist-test-slave-9gcw"
server_key: "cbcd1a98ad6eabb7890cad975be074fe"
server_key_iv: "fb20a1ecb19e9aa105735f91ce9898bd"
server_key_version: "encryptionkey@0"
I20260811 01:50:18.256062 15265 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 01:50:18.259876 15281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:18.261660 15265 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:50:18.261922 15265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "bdd5a45b7eeb4324a3a728ad0e62b2e5"
format_stamp: "Formatted at 2026-08-11 01:50:18 on dist-test-slave-9gcw"
server_key: "cbcd1a98ad6eabb7890cad975be074fe"
server_key_iv: "fb20a1ecb19e9aa105735f91ce9898bd"
server_key_version: "encryptionkey@0"
I20260811 01:50:18.262173 15265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:18.301553 15265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:18.307035 15265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:18.307374 15265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:18.327817 15265 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:38755
I20260811 01:50:18.327841 15338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:38755 every 8 connection(s)
I20260811 01:50:18.329406 15265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:50:18.333575 15339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:50:18.336591 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 15265
I20260811 01:50:18.336769 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:50:18.337381 22361 external_mini_cluster.cc:1428] Setting key e1e730b28744819da32687bd71ca5ed4
I20260811 01:50:18.346638 15339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5: Bootstrap starting.
I20260811 01:50:18.350665 15339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:18.351971 15339 log.cc:824] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5: Log is configured to *not* fsync() on all Append() calls
I20260811 01:50:18.353024 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:18.339505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38155 (local address 127.21.214.126:38755)
0811 01:50:18.340420 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:18.340433 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:18.340480 (+ 47us) server_negotiation.cc:407] Connection header received
0811 01:50:18.342398 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:18.342425 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:18.342891 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:18.343255 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:18.343941 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:18.345443 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:18.347358 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:18.348154 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:18.348315 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:18.350058 (+ 1743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:18.350066 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:18.350083 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:50:18.350673 (+ 590us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:18.350869 (+ 196us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:50:18.350909 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:18.351047 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:18.351404 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 01:50:18.351612 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":422,"threads_started":1}
I20260811 01:50:18.355494 15339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5: No bootstrap required, opened a new log
I20260811 01:50:18.357230 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:50:18.365908 15339 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd5a45b7eeb4324a3a728ad0e62b2e5" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 38755 } }
I20260811 01:50:18.366479 15339 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:50:18.366680 15339 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdd5a45b7eeb4324a3a728ad0e62b2e5, State: Initialized, Role: FOLLOWER
I20260811 01:50:18.367460 15339 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd5a45b7eeb4324a3a728ad0e62b2e5" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 38755 } }
I20260811 01:50:18.367784 15339 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:50:18.367980 15339 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:50:18.368183 15339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:50:18.369436 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:18.362075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47403 (local address 127.21.214.126:38755)
0811 01:50:18.362327 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:18.362339 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:18.362592 (+ 253us) server_negotiation.cc:407] Connection header received
0811 01:50:18.362770 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:18.362781 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:18.362879 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:18.363012 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:18.363578 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:18.364937 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:18.366479 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:18.367110 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:18.367251 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:18.368584 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:18.368589 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:18.368593 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:50:18.368612 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:18.368653 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:50:18.368670 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:18.368809 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:18.369027 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 01:50:18.369088 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 01:50:18.372007 15339 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd5a45b7eeb4324a3a728ad0e62b2e5" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 38755 } }
I20260811 01:50:18.372730 15339 leader_election.cc:302] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdd5a45b7eeb4324a3a728ad0e62b2e5; no voters:
I20260811 01:50:18.373490 15339 leader_election.cc:288] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:50:18.373641 15350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:50:18.375284 15350 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5 [term 1 LEADER]: Becoming Leader. State: Replica: bdd5a45b7eeb4324a3a728ad0e62b2e5, State: Running, Role: LEADER
I20260811 01:50:18.375890 15350 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd5a45b7eeb4324a3a728ad0e62b2e5" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 38755 } }
I20260811 01:50:18.376085 15339 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:50:18.379001 15352 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdd5a45b7eeb4324a3a728ad0e62b2e5. Latest consensus state: current_term: 1 leader_uuid: "bdd5a45b7eeb4324a3a728ad0e62b2e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd5a45b7eeb4324a3a728ad0e62b2e5" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 38755 } } }
I20260811 01:50:18.379443 15352 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:18.379997 15351 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdd5a45b7eeb4324a3a728ad0e62b2e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd5a45b7eeb4324a3a728ad0e62b2e5" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 38755 } } }
I20260811 01:50:18.380323 15351 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:18.382215 15361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:50:18.387862 15361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:50:18.399241 15361 catalog_manager.cc:1347] Generated new cluster ID: f5736cf54ddc42edb0c6c2eb05e52bc7
I20260811 01:50:18.399382 15361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:50:18.438570 15361 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:50:18.439996 15361 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:50:18.446938 15361 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdd5a45b7eeb4324a3a728ad0e62b2e5: Generated new TSK 0
I20260811 01:50:18.448186 15361 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:50:18.454790 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 15265
W20260811 01:50:18.456864 15334 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38155
W20260811 01:50:18.457758 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 15265: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:50:18.457938 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 15265
W20260811 01:50:18.482831 15249 connection.cc:535] client connection to 127.21.214.126:38755 recv error: Network error: failed to read from TLS socket (remote: 127.21.214.126:38755): Connection reset by peer (error 104)
2026-08-11T01:50:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2233 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:50:18 dist-test-slave-9gcw krb5kdc[15377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:50:18 dist-test-slave-9gcw krb5kdc[15377](info): set up 2 sockets
Aug 11 01:50:18 dist-test-slave-9gcw krb5kdc[15377](info): commencing operation
krb5kdc: starting...
W20260811 01:50:19.754988 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:50:19 dist-test-slave-9gcw krb5kdc[15377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:50:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:50:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:50:20.117403 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:43491
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:41069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:43491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:50:20.484299 15395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:20.484638 15395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:20.484827 15395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:20.492352 15395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:20.492492 15395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:20.492594 15395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:20.492707 15395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:20.492810 15395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:20.503499 15395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:41069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:43491
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:43491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:20.504957 15395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:20.506361 15395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:20.518952 15401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:20.518893 15402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:20.518996 15404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:20.520274 15395 server_base.cc:1034] running on GCE node
I20260811 01:50:20.521095 15395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:20.522769 15395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:20.523984 15395 hybrid_clock.cc:648] HybridClock initialized: now 1786413020523928 us; error 35 us; skew 500 ppm
I20260811 01:50:20.524358 15395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:20 dist-test-slave-9gcw krb5kdc[15377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413020, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:20.527591 15395 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:50:20.529239 15395 webserver.cc:460] Webserver started at http://127.21.214.126:35099/ using document root <none> and password file <none>
I20260811 01:50:20.529978 15395 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:20.530109 15395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:20.530438 15395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:20.533605 15395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "e682b4d95ab24d97b498b5a8a4d2f3c8"
format_stamp: "Formatted at 2026-08-11 01:50:20 on dist-test-slave-9gcw"
server_key: "45fa5f650558087b7195902eeb58fa0a"
server_key_iv: "2333b7a470adea3b27271c9a0ca5e493"
server_key_version: "encryptionkey@0"
I20260811 01:50:20.534422 15395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "e682b4d95ab24d97b498b5a8a4d2f3c8"
format_stamp: "Formatted at 2026-08-11 01:50:20 on dist-test-slave-9gcw"
server_key: "45fa5f650558087b7195902eeb58fa0a"
server_key_iv: "2333b7a470adea3b27271c9a0ca5e493"
server_key_version: "encryptionkey@0"
I20260811 01:50:20.540042 15395 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 01:50:20.543743 15412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:20.545392 15395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 01:50:20.545601 15395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "e682b4d95ab24d97b498b5a8a4d2f3c8"
format_stamp: "Formatted at 2026-08-11 01:50:20 on dist-test-slave-9gcw"
server_key: "45fa5f650558087b7195902eeb58fa0a"
server_key_iv: "2333b7a470adea3b27271c9a0ca5e493"
server_key_version: "encryptionkey@0"
I20260811 01:50:20.545840 15395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:20.552050 15395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:20.556564 15395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:20.556908 15395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:20.576668 15395 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:43491
I20260811 01:50:20.576700 15468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:43491 every 8 connection(s)
I20260811 01:50:20.578135 15395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:50:20.582484 15469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:50:20.585988 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 15395
I20260811 01:50:20.586201 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:50:20.586771 22361 external_mini_cluster.cc:1428] Setting key 6fd0754f2f7222515bbfba04c172d020
I20260811 01:50:20.597748 15472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:20.589003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35289 (local address 127.21.214.126:43491)
0811 01:50:20.589977 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:20.589994 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:20.590421 (+ 427us) server_negotiation.cc:407] Connection header received
0811 01:50:20.592675 (+ 2254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:20.592715 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:20.593264 (+ 549us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:20.593872 (+ 608us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:20.595287 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:20.595299 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:20.595321 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:50:20.595358 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:20.595463 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:50:20.595497 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:20.595774 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:20.595937 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 01:50:20.596179 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":404,"threads_started":1}
I20260811 01:50:20.598423 15469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8: Bootstrap starting.
I20260811 01:50:20.601985 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:50:20.602465 15469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:20.603891 15469 log.cc:824] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8: Log is configured to *not* fsync() on all Append() calls
I20260811 01:50:20.607848 15469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8: No bootstrap required, opened a new log
I20260811 01:50:20.610749 15472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:20.607965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58097 (local address 127.21.214.126:43491)
0811 01:50:20.608169 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:20.608176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:20.608583 (+ 407us) server_negotiation.cc:407] Connection header received
0811 01:50:20.608742 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:20.608749 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:20.608819 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:20.608960 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:20.609943 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:20.609950 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:20.609955 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:50:20.609982 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:20.610032 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:50:20.610057 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:20.610241 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:20.610309 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 01:50:20.610378 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 01:50:20.618170 15469 raft_consensus.cc:348] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e682b4d95ab24d97b498b5a8a4d2f3c8" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43491 } }
I20260811 01:50:20.618644 15469 raft_consensus.cc:374] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:50:20.618736 15469 raft_consensus.cc:729] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e682b4d95ab24d97b498b5a8a4d2f3c8, State: Initialized, Role: FOLLOWER
I20260811 01:50:20.619532 15469 consensus_queue.cc:260] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e682b4d95ab24d97b498b5a8a4d2f3c8" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43491 } }
I20260811 01:50:20.619825 15469 raft_consensus.cc:388] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:50:20.620016 15469 raft_consensus.cc:482] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:50:20.620278 15469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:50:20.623027 15469 raft_consensus.cc:504] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e682b4d95ab24d97b498b5a8a4d2f3c8" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43491 } }
I20260811 01:50:20.623636 15469 leader_election.cc:302] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e682b4d95ab24d97b498b5a8a4d2f3c8; no voters:
I20260811 01:50:20.624297 15469 leader_election.cc:288] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:50:20.624464 15483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:50:20.625319 15483 raft_consensus.cc:686] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8 [term 1 LEADER]: Becoming Leader. State: Replica: e682b4d95ab24d97b498b5a8a4d2f3c8, State: Running, Role: LEADER
I20260811 01:50:20.625919 15483 consensus_queue.cc:237] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e682b4d95ab24d97b498b5a8a4d2f3c8" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43491 } }
I20260811 01:50:20.626909 15469 sys_catalog.cc:564] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:50:20.629382 15484 sys_catalog.cc:455] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e682b4d95ab24d97b498b5a8a4d2f3c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e682b4d95ab24d97b498b5a8a4d2f3c8" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43491 } } }
I20260811 01:50:20.628959 15485 sys_catalog.cc:455] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e682b4d95ab24d97b498b5a8a4d2f3c8. Latest consensus state: current_term: 1 leader_uuid: "e682b4d95ab24d97b498b5a8a4d2f3c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e682b4d95ab24d97b498b5a8a4d2f3c8" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 43491 } } }
I20260811 01:50:20.629814 15484 sys_catalog.cc:458] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:20.629854 15485 sys_catalog.cc:458] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:20.630905 15490 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:50:20.637410 15490 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:50:20.648797 15490 catalog_manager.cc:1347] Generated new cluster ID: cce782f87fd74bdabecbf8d648f2c5a4
I20260811 01:50:20.648974 15490 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:50:20.674338 15490 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:50:20.675027 15490 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:50:20.685058 15490 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e682b4d95ab24d97b498b5a8a4d2f3c8: Generated new TSK 0
I20260811 01:50:20.686138 15490 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:50:20.696264 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 15395
W20260811 01:50:20.697672 15464 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35289
W20260811 01:50:20.698501 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 15395: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:50:20.698679 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 15395
2026-08-11T01:50:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2239 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:50:20 dist-test-slave-9gcw krb5kdc[15509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:50:20 dist-test-slave-9gcw krb5kdc[15509](info): set up 2 sockets
Aug 11 01:50:20 dist-test-slave-9gcw krb5kdc[15509](info): commencing operation
krb5kdc: starting...
W20260811 01:50:21.985280 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:50:22 dist-test-slave-9gcw krb5kdc[15509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:50:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:50:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:50:22.349738 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:45241
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:44929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:45241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:50:22.718716 15522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:22.719030 15522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:22.719197 15522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:22.726739 15522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:22.726871 15522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:22.726963 15522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:22.727090 15522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:22.727160 15522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:22.737573 15522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:44929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:45241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:45241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:22.739009 15522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:22.740454 15522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:22.753445 15528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:22.753482 15527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:22.753988 15531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:22.754366 15522 server_base.cc:1034] running on GCE node
I20260811 01:50:22.755249 15522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:22.756803 15522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:22.758065 15522 hybrid_clock.cc:648] HybridClock initialized: now 1786413022757989 us; error 66 us; skew 500 ppm
I20260811 01:50:22.758471 15522 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:22 dist-test-slave-9gcw krb5kdc[15509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413022, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:22.769452 15522 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:50:22.771268 15522 webserver.cc:460] Webserver started at http://127.21.214.126:43265/ using document root <none> and password file <none>
I20260811 01:50:22.772085 15522 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:22.772246 15522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:22.772580 15522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:22.776064 15522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "1d6faa1e4d6642728c5f3105fac0776e"
format_stamp: "Formatted at 2026-08-11 01:50:22 on dist-test-slave-9gcw"
server_key: "5679d4ac6d2c6dff48115fc7ea6ee608"
server_key_iv: "77053c3744942035285a4ad5e20c8325"
server_key_version: "encryptionkey@0"
I20260811 01:50:22.776908 15522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "1d6faa1e4d6642728c5f3105fac0776e"
format_stamp: "Formatted at 2026-08-11 01:50:22 on dist-test-slave-9gcw"
server_key: "5679d4ac6d2c6dff48115fc7ea6ee608"
server_key_iv: "77053c3744942035285a4ad5e20c8325"
server_key_version: "encryptionkey@0"
I20260811 01:50:22.782698 15522 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 01:50:22.786593 15540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:22.788089 15522 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:50:22.788295 15522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "1d6faa1e4d6642728c5f3105fac0776e"
format_stamp: "Formatted at 2026-08-11 01:50:22 on dist-test-slave-9gcw"
server_key: "5679d4ac6d2c6dff48115fc7ea6ee608"
server_key_iv: "77053c3744942035285a4ad5e20c8325"
server_key_version: "encryptionkey@0"
I20260811 01:50:22.788515 15522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:22.821574 15522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:22.826596 15522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:22.826944 15522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:22.847230 15522 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:45241
I20260811 01:50:22.847249 15603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:45241 every 8 connection(s)
I20260811 01:50:22.848749 15522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:50:22.852944 15604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:50:22.856914 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 15522
I20260811 01:50:22.857110 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:50:22.857632 22361 external_mini_cluster.cc:1428] Setting key 7c53fe86470647d5623b75edc044cc22
I20260811 01:50:22.865976 15604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e: Bootstrap starting.
I20260811 01:50:22.869362 15604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:22.870443 15604 log.cc:824] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e: Log is configured to *not* fsync() on all Append() calls
Aug 11 01:50:22 dist-test-slave-9gcw krb5kdc[15509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:50:22.873950 15604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e: No bootstrap required, opened a new log
I20260811 01:50:22.882895 15607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:22.859573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38409 (local address 127.21.214.126:45241)
0811 01:50:22.860414 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:22.860428 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:22.860485 (+ 57us) server_negotiation.cc:407] Connection header received
0811 01:50:22.862375 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:22.862400 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:22.862873 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:22.863291 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:22.864807 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:22.866283 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:22.867620 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:22.868338 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:22.868540 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:22.873122 (+ 4582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:22.873133 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:22.873185 (+ 52us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:22.873232 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:22.877663 (+ 4431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:22.878501 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:22.878511 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:22.878520 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:22.878763 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:22.879157 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:22.879162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:22.879166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:22.880006 (+ 840us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:22.880335 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:22.880764 (+ 429us) server_negotiation.cc:299] Negotiation successful
0811 01:50:22.881322 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":381,"threads_started":1}
I20260811 01:50:22.885393 15604 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6faa1e4d6642728c5f3105fac0776e" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45241 } }
I20260811 01:50:22.885897 15604 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:50:22.886034 15604 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d6faa1e4d6642728c5f3105fac0776e, State: Initialized, Role: FOLLOWER
I20260811 01:50:22.886678 15604 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6faa1e4d6642728c5f3105fac0776e" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45241 } }
I20260811 01:50:22.886934 15604 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:50:22.887104 15604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:50:22.887339 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:50:22.887326 15604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:50:22.890739 15604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6faa1e4d6642728c5f3105fac0776e" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45241 } }
I20260811 01:50:22.891604 15604 leader_election.cc:302] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d6faa1e4d6642728c5f3105fac0776e; no voters:
I20260811 01:50:22.892314 15604 leader_election.cc:288] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:50:22.892472 15614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:50:22.893306 15614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e [term 1 LEADER]: Becoming Leader. State: Replica: 1d6faa1e4d6642728c5f3105fac0776e, State: Running, Role: LEADER
I20260811 01:50:22.893975 15614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6faa1e4d6642728c5f3105fac0776e" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45241 } }
I20260811 01:50:22.897212 15604 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:50:22.897467 15615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d6faa1e4d6642728c5f3105fac0776e. Latest consensus state: current_term: 1 leader_uuid: "1d6faa1e4d6642728c5f3105fac0776e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6faa1e4d6642728c5f3105fac0776e" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45241 } } }
I20260811 01:50:22.897897 15615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:22.898438 15617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d6faa1e4d6642728c5f3105fac0776e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6faa1e4d6642728c5f3105fac0776e" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 45241 } } }
I20260811 01:50:22.898733 15617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:22.900799 15620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:50:22.907399 15620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:50:22.917330 15607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:22.893636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52399 (local address 127.21.214.126:45241)
0811 01:50:22.893844 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:22.893856 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:22.894621 (+ 765us) server_negotiation.cc:407] Connection header received
0811 01:50:22.895006 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:22.895015 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:22.895125 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:22.895245 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:22.897505 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:22.898770 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:22.901221 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:22.902001 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:22.902146 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:22.906470 (+ 4324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:22.906486 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:22.906494 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:22.906552 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:22.911618 (+ 5066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:22.912898 (+ 1280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:22.912908 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:22.912914 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:22.913114 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:22.913739 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:22.913747 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:22.913753 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:22.914376 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:22.914707 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:22.916400 (+ 1693us) server_negotiation.cc:299] Negotiation successful
0811 01:50:22.916729 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 01:50:22.932542 15620 catalog_manager.cc:1347] Generated new cluster ID: 42f03ce8bf894a87a75c43b8bcf218b7
I20260811 01:50:22.932802 15620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:50:22.954653 15620 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:50:22.956332 15620 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:50:22.972425 15620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d6faa1e4d6642728c5f3105fac0776e: Generated new TSK 0
I20260811 01:50:22.973624 15620 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:50:22.991515 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 15522
I20260811 01:50:22.992797 15599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:50:23.093389 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 15522
2026-08-11T01:50:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2395 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:50:23 dist-test-slave-9gcw krb5kdc[15641](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:50:23 dist-test-slave-9gcw krb5kdc[15641](info): set up 2 sockets
Aug 11 01:50:23 dist-test-slave-9gcw krb5kdc[15641](info): commencing operation
krb5kdc: starting...
W20260811 01:50:24.363265 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:50:24 dist-test-slave-9gcw krb5kdc[15641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:50:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:50:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:50:24.721011 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:36039
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:43053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:36039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:50:25.088985 15661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:25.089404 15661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:25.089584 15661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:25.097069 15661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:25.097289 15661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:25.097396 15661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:25.097513 15661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:25.097615 15661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:25.107942 15661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:43053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:36039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:36039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:25.109498 15661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:25.111133 15661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:25.124192 15666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:25.124284 15668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:25.124291 15670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:25.125799 15661 server_base.cc:1034] running on GCE node
I20260811 01:50:25.126711 15661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:25.128268 15661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:25.129508 15661 hybrid_clock.cc:648] HybridClock initialized: now 1786413025129456 us; error 46 us; skew 500 ppm
I20260811 01:50:25.129884 15661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:25 dist-test-slave-9gcw krb5kdc[15641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413025, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:25.133400 15661 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:50:25.135087 15661 webserver.cc:460] Webserver started at http://127.21.214.126:34089/ using document root <none> and password file <none>
I20260811 01:50:25.135919 15661 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:25.136034 15661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:25.136317 15661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:25.139588 15661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "1b452faeab9b4739aede3a05a5ca4448"
format_stamp: "Formatted at 2026-08-11 01:50:25 on dist-test-slave-9gcw"
server_key: "37503d5f7de810403ffab78d8b686234"
server_key_iv: "9d3db2d9e3d17e63f0c387d228ac8090"
server_key_version: "encryptionkey@0"
I20260811 01:50:25.140436 15661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "1b452faeab9b4739aede3a05a5ca4448"
format_stamp: "Formatted at 2026-08-11 01:50:25 on dist-test-slave-9gcw"
server_key: "37503d5f7de810403ffab78d8b686234"
server_key_iv: "9d3db2d9e3d17e63f0c387d228ac8090"
server_key_version: "encryptionkey@0"
I20260811 01:50:25.146389 15661 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 01:50:25.150264 15677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:25.151825 15661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 01:50:25.151991 15661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "1b452faeab9b4739aede3a05a5ca4448"
format_stamp: "Formatted at 2026-08-11 01:50:25 on dist-test-slave-9gcw"
server_key: "37503d5f7de810403ffab78d8b686234"
server_key_iv: "9d3db2d9e3d17e63f0c387d228ac8090"
server_key_version: "encryptionkey@0"
I20260811 01:50:25.152185 15661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:25.164227 15661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:25.169006 15661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:25.169371 15661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:25.189817 15661 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:36039
I20260811 01:50:25.189883 15736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:36039 every 8 connection(s)
I20260811 01:50:25.191428 15661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:50:25.195717 15737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:50:25.196640 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 15661
I20260811 01:50:25.196879 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:50:25.197467 22361 external_mini_cluster.cc:1428] Setting key 1d7a177557c23a6a15d09da7a142481e
I20260811 01:50:25.211290 15737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448: Bootstrap starting.
I20260811 01:50:25.213296 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:25.199554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41495 (local address 127.21.214.126:36039)
0811 01:50:25.200399 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:25.200416 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:25.200477 (+ 61us) server_negotiation.cc:407] Connection header received
0811 01:50:25.202857 (+ 2380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:25.202891 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:25.203506 (+ 615us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:25.204046 (+ 540us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:25.204598 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:25.206553 (+ 1955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:25.207915 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:25.209104 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:25.209342 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:25.210699 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:25.210707 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:25.210724 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:50:25.211293 (+ 569us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:25.211420 (+ 127us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:50:25.211476 (+ 56us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:25.211631 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:25.211898 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 01:50:25.212189 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":341,"threads_started":1}
I20260811 01:50:25.215214 15737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:25.216547 15737 log.cc:824] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448: Log is configured to *not* fsync() on all Append() calls
I20260811 01:50:25.217464 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:50:25.219529 15737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448: No bootstrap required, opened a new log
I20260811 01:50:25.229022 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:25.222193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50263 (local address 127.21.214.126:36039)
0811 01:50:25.222396 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:25.222408 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:25.222843 (+ 435us) server_negotiation.cc:407] Connection header received
0811 01:50:25.222980 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:25.222991 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:25.223072 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:25.223191 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:25.223716 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:25.224833 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:25.226292 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:25.227037 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:25.227173 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:25.228260 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:25.228267 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:25.228272 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:50:25.228295 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:25.228348 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:50:25.228368 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:25.228468 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:25.228611 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 01:50:25.228716 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 01:50:25.229576 15737 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b452faeab9b4739aede3a05a5ca4448" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36039 } }
I20260811 01:50:25.229990 15737 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:50:25.230129 15737 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b452faeab9b4739aede3a05a5ca4448, State: Initialized, Role: FOLLOWER
I20260811 01:50:25.230805 15737 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b452faeab9b4739aede3a05a5ca4448" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36039 } }
I20260811 01:50:25.231066 15737 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:50:25.231235 15737 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:50:25.231420 15737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:50:25.233862 15737 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b452faeab9b4739aede3a05a5ca4448" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36039 } }
I20260811 01:50:25.234416 15737 leader_election.cc:302] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b452faeab9b4739aede3a05a5ca4448; no voters:
I20260811 01:50:25.235080 15737 leader_election.cc:288] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:50:25.235293 15750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:50:25.236513 15750 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448 [term 1 LEADER]: Becoming Leader. State: Replica: 1b452faeab9b4739aede3a05a5ca4448, State: Running, Role: LEADER
I20260811 01:50:25.237138 15750 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b452faeab9b4739aede3a05a5ca4448" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36039 } }
I20260811 01:50:25.237534 15737 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:50:25.241058 15751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b452faeab9b4739aede3a05a5ca4448" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b452faeab9b4739aede3a05a5ca4448" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36039 } } }
I20260811 01:50:25.241160 15752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b452faeab9b4739aede3a05a5ca4448. Latest consensus state: current_term: 1 leader_uuid: "1b452faeab9b4739aede3a05a5ca4448" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b452faeab9b4739aede3a05a5ca4448" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 36039 } } }
I20260811 01:50:25.241551 15751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:25.241573 15752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:25.244014 15757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:50:25.250187 15757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:50:25.261005 15757 catalog_manager.cc:1347] Generated new cluster ID: 37a632c72f8947f2a68d7dbd58bf962e
I20260811 01:50:25.261189 15757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:50:25.283290 15757 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:50:25.285133 15757 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:50:25.296419 15757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b452faeab9b4739aede3a05a5ca4448: Generated new TSK 0
I20260811 01:50:25.297417 15757 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:50:25.312260 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 15661
W20260811 01:50:25.313817 15731 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41495
W20260811 01:50:25.314620 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 15661: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:50:25.314800 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 15661
2026-08-11T01:50:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2220 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:50:25 dist-test-slave-9gcw krb5kdc[15777](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:50:25 dist-test-slave-9gcw krb5kdc[15777](info): set up 2 sockets
Aug 11 01:50:25 dist-test-slave-9gcw krb5kdc[15777](info): commencing operation
krb5kdc: starting...
W20260811 01:50:26.569242 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:50:26 dist-test-slave-9gcw krb5kdc[15777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:50:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:50:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:50:26.933542 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:39785
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:34835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:39785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:50:27.320884 15792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:27.321357 15792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:27.321569 15792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:27.329572 15792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:27.329783 15792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:27.329882 15792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:27.329989 15792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:27.330066 15792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:27.340986 15792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:34835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:39785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:39785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:27.342613 15792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:27.344316 15792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:27.358337 15800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:27.358297 15804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:27.358289 15801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:27.359794 15792 server_base.cc:1034] running on GCE node
I20260811 01:50:27.360658 15792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:27.362500 15792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:27.363741 15792 hybrid_clock.cc:648] HybridClock initialized: now 1786413027363697 us; error 35 us; skew 500 ppm
I20260811 01:50:27.364144 15792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:27 dist-test-slave-9gcw krb5kdc[15777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413027, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:27.367643 15792 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:50:27.369438 15792 webserver.cc:460] Webserver started at http://127.21.214.126:41119/ using document root <none> and password file <none>
I20260811 01:50:27.370204 15792 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:27.370359 15792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:27.370689 15792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:27.373939 15792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "d78dfe004b93422e867eab804714bc51"
format_stamp: "Formatted at 2026-08-11 01:50:27 on dist-test-slave-9gcw"
server_key: "bf9a3bd6fc16a9b221250f2937c27f20"
server_key_iv: "ba0367fcf1b5119d8d826ac6e25a823d"
server_key_version: "encryptionkey@0"
I20260811 01:50:27.374765 15792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "d78dfe004b93422e867eab804714bc51"
format_stamp: "Formatted at 2026-08-11 01:50:27 on dist-test-slave-9gcw"
server_key: "bf9a3bd6fc16a9b221250f2937c27f20"
server_key_iv: "ba0367fcf1b5119d8d826ac6e25a823d"
server_key_version: "encryptionkey@0"
I20260811 01:50:27.380416 15792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 01:50:27.384262 15812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:27.385885 15792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 01:50:27.386088 15792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "d78dfe004b93422e867eab804714bc51"
format_stamp: "Formatted at 2026-08-11 01:50:27 on dist-test-slave-9gcw"
server_key: "bf9a3bd6fc16a9b221250f2937c27f20"
server_key_iv: "ba0367fcf1b5119d8d826ac6e25a823d"
server_key_version: "encryptionkey@0"
I20260811 01:50:27.386307 15792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:27.392637 15792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:27.397379 15792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:27.397714 15792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:27.417723 15792 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:39785
I20260811 01:50:27.417779 15866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:39785 every 8 connection(s)
I20260811 01:50:27.419360 15792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:50:27.423297 15867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:50:27.429782 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 15792
I20260811 01:50:27.430001 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:50:27.430508 22361 external_mini_cluster.cc:1428] Setting key 95b011fcd63c83980b0f25031de8550a
I20260811 01:50:27.434834 15867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51: Bootstrap starting.
I20260811 01:50:27.438812 15867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:27.438963 15871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:27.432436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50345 (local address 127.21.214.126:39785)
0811 01:50:27.433365 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:27.433377 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:27.433423 (+ 46us) server_negotiation.cc:407] Connection header received
0811 01:50:27.435183 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:27.435209 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:27.435588 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:27.435939 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:27.437236 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:27.437243 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:27.437261 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:50:27.437277 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:27.437347 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:50:27.437371 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:27.437539 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:27.437690 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 01:50:27.437933 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":407,"threads_started":1}
I20260811 01:50:27.440310 15867 log.cc:824] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51: Log is configured to *not* fsync() on all Append() calls
I20260811 01:50:27.443271 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:50:27.443768 15867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51: No bootstrap required, opened a new log
I20260811 01:50:27.450523 15871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:27.447779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48121 (local address 127.21.214.126:39785)
0811 01:50:27.447965 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:27.447973 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:27.448490 (+ 517us) server_negotiation.cc:407] Connection header received
0811 01:50:27.448630 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:27.448636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:27.448708 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:27.448831 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:27.449808 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:27.449814 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:27.449820 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 01:50:27.449845 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:27.449893 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 01:50:27.449914 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:27.450051 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:27.450125 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 01:50:27.450188 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 01:50:27.453054 15867 raft_consensus.cc:348] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d78dfe004b93422e867eab804714bc51" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 39785 } }
I20260811 01:50:27.453547 15867 raft_consensus.cc:374] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 01:50:27.453680 15867 raft_consensus.cc:729] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d78dfe004b93422e867eab804714bc51, State: Initialized, Role: FOLLOWER
I20260811 01:50:27.454273 15867 consensus_queue.cc:260] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d78dfe004b93422e867eab804714bc51" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 39785 } }
I20260811 01:50:27.454532 15867 raft_consensus.cc:388] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 01:50:27.454700 15867 raft_consensus.cc:482] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 01:50:27.454887 15867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:50:27.457245 15867 raft_consensus.cc:504] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d78dfe004b93422e867eab804714bc51" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 39785 } }
I20260811 01:50:27.457799 15867 leader_election.cc:302] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d78dfe004b93422e867eab804714bc51; no voters:
I20260811 01:50:27.458667 15867 leader_election.cc:288] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 01:50:27.458793 15881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:50:27.459515 15881 raft_consensus.cc:686] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51 [term 1 LEADER]: Becoming Leader. State: Replica: d78dfe004b93422e867eab804714bc51, State: Running, Role: LEADER
I20260811 01:50:27.460192 15881 consensus_queue.cc:237] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d78dfe004b93422e867eab804714bc51" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 39785 } }
I20260811 01:50:27.461083 15867 sys_catalog.cc:564] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:50:27.463567 15882 sys_catalog.cc:455] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d78dfe004b93422e867eab804714bc51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d78dfe004b93422e867eab804714bc51" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 39785 } } }
I20260811 01:50:27.463896 15883 sys_catalog.cc:455] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d78dfe004b93422e867eab804714bc51. Latest consensus state: current_term: 1 leader_uuid: "d78dfe004b93422e867eab804714bc51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d78dfe004b93422e867eab804714bc51" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 39785 } } }
I20260811 01:50:27.464491 15882 sys_catalog.cc:458] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:27.464504 15883 sys_catalog.cc:458] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:27.470762 15891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:50:27.476564 15891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:50:27.486665 15891 catalog_manager.cc:1347] Generated new cluster ID: 4624d441d9b34d2b963f524ff4511d2f
I20260811 01:50:27.486811 15891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:50:27.496488 15891 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:50:27.497043 15891 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:50:27.505293 15891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d78dfe004b93422e867eab804714bc51: Generated new TSK 0
I20260811 01:50:27.506246 15891 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 01:50:27.511672 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 15792
W20260811 01:50:27.513104 15862 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50345
W20260811 01:50:27.513887 22361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 15792: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 01:50:27.514056 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 15792
2026-08-11T01:50:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2199 ms)
[----------] 18 tests from AuthTokenIssuingTest (41007 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:50:27 dist-test-slave-9gcw krb5kdc[15907](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:50:27 dist-test-slave-9gcw krb5kdc[15907](info): set up 2 sockets
Aug 11 01:50:27 dist-test-slave-9gcw krb5kdc[15907](info): commencing operation
krb5kdc: starting...
W20260811 01:50:28.854790 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:50:29 dist-test-slave-9gcw krb5kdc[15907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:50:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:50:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:50:29.230136 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:34813
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:41571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:34813,127.21.214.125:44281,127.21.214.124:45993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:50:29.602607 15923 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 01:50:29.603042 15923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:29.603180 15923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:29.603332 15923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:29.611068 15923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:29.611200 15923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:29.611316 15923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:29.611418 15923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:29.611518 15923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:29.622469 15923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:41571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:34813,127.21.214.125:44281,127.21.214.124:45993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:34813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:29.624015 15923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:29.625749 15923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:29.639369 15931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:29.639410 15929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:29.639369 15928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:29.640357 15923 server_base.cc:1034] running on GCE node
I20260811 01:50:29.641356 15923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:29.643038 15923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:29.644352 15923 hybrid_clock.cc:648] HybridClock initialized: now 1786413029644265 us; error 56 us; skew 500 ppm
I20260811 01:50:29.644805 15923 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:29 dist-test-slave-9gcw krb5kdc[15907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413029, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:29.652969 15923 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:50:29.654834 15923 webserver.cc:460] Webserver started at http://127.21.214.126:39313/ using document root <none> and password file <none>
I20260811 01:50:29.655684 15923 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:29.655841 15923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:29.656214 15923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:29.659622 15923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4"
format_stamp: "Formatted at 2026-08-11 01:50:29 on dist-test-slave-9gcw"
server_key: "152139d5bc11f9e078cc256cca0a0432"
server_key_iv: "166e47343ca97b82d4ce18c230965d48"
server_key_version: "encryptionkey@0"
I20260811 01:50:29.660524 15923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4"
format_stamp: "Formatted at 2026-08-11 01:50:29 on dist-test-slave-9gcw"
server_key: "152139d5bc11f9e078cc256cca0a0432"
server_key_iv: "166e47343ca97b82d4ce18c230965d48"
server_key_version: "encryptionkey@0"
I20260811 01:50:29.666543 15923 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 01:50:29.670647 15940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:29.672240 15923 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 01:50:29.672461 15923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4"
format_stamp: "Formatted at 2026-08-11 01:50:29 on dist-test-slave-9gcw"
server_key: "152139d5bc11f9e078cc256cca0a0432"
server_key_iv: "166e47343ca97b82d4ce18c230965d48"
server_key_version: "encryptionkey@0"
I20260811 01:50:29.672727 15923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:29.697840 15923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:29.702826 15923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:29.703156 15923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:29.723475 15923 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:34813
I20260811 01:50:29.723482 15998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:34813 every 8 connection(s)
I20260811 01:50:29.724977 15923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:50:29.725698 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 15923
I20260811 01:50:29.725981 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:50:29.726830 22361 external_mini_cluster.cc:1428] Setting key 3f0b13ff963bd3ca52e60f46e0202e18
I20260811 01:50:29.729427 15999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:50:29.736268 15999 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:50:29 dist-test-slave-9gcw krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413029, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:50:29.760869 16004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:29.741116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54646 (local address 127.21.214.126:34813)
0811 01:50:29.741935 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:29.741941 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:29.741986 (+ 45us) server_negotiation.cc:407] Connection header received
0811 01:50:29.743647 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:29.743653 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:29.743895 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:29.744127 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:29.745563 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:29.746880 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:29.748334 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:29.748895 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:29.749111 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:29.752946 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:29.752955 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:29.752969 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:29.753003 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:29.755978 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:29.757033 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:29.757040 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:29.757042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:29.757264 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:29.757823 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:29.757827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:29.757829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:29.758382 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:29.758549 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:29.759297 (+ 748us) server_negotiation.cc:299] Negotiation successful
0811 01:50:29.759653 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":633,"thread_start_us":394,"threads_started":1}
I20260811 01:50:29.760926 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:29.739880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:34813 (local address 127.0.0.1:54646)
0811 01:50:29.740809 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:29.740868 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 01:50:29.743283 (+ 2415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:50:29.744138 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:50:29.744144 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:50:29.744915 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:50:29.745335 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:29.745356 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:29.747053 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:29.747063 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:29.748181 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:29.748193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:29.749134 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:29.749176 (+ 42us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:29.749389 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:29.750088 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:50:29.750111 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:50:29.752706 (+ 2595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:50:29.756159 (+ 3453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:29.756176 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:29.756193 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:29.756873 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:29.757398 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:29.757406 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:29.757412 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:29.757678 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:29.758540 (+ 862us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:50:29.758552 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:50:29.758833 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 01:50:29.759149 (+ 316us) client_negotiation.cc:240] Negotiation successful
0811 01:50:29.759861 (+ 712us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":85,"thread_start_us":341,"threads_started":1}
WARNING: no policy specified for kudu/127.21.214.125@KRBTEST.COM; defaulting to no policy
I20260811 01:50:29.764101 15999 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.125@KRBTEST.COM" created.
I20260811 01:50:29.765578 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:29.765093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:44281 (local address 127.0.0.1:43186)
0811 01:50:29.765341 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:29.765457 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:44281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260811 01:50:29.766053 15944 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.214.125:44281: connect: Connection refused (error 111)
I20260811 01:50:29.767745 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:29.767383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:44281 (local address 127.0.0.1:43190)
0811 01:50:29.767547 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:29.767618 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:44281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 01:50:29.768050 15999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.125:44281: Network error: Client connection negotiation failed: client connection to 127.21.214.125:44281: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.125.keytab.
Entry for principal kudu/127.21.214.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.125.keytab.
I20260811 01:50:29.804859 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.125:44281
--webserver_interface=127.21.214.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.125.keytab
--principal=kudu/127.21.214.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:41571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:34813,127.21.214.125:44281,127.21.214.124:45993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:50:29.817440 15999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } attempt: 1
I20260811 01:50:29.818877 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:29.818245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:44281 (local address 127.0.0.1:43198)
0811 01:50:29.818524 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:29.818689 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:44281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 01:50:29.820117 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:29.819673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:44281 (local address 127.0.0.1:43214)
0811 01:50:29.819844 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:29.819938 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:44281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 01:50:29.820329 15999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.125:44281: Network error: Client connection negotiation failed: client connection to 127.21.214.125:44281: connect: Connection refused (error 111)
I20260811 01:50:29.888624 15999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } attempt: 2
I20260811 01:50:29.890007 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:29.889418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:44281 (local address 127.0.0.1:43230)
0811 01:50:29.889704 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:29.889823 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:44281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 01:50:29.891359 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:29.890883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:44281 (local address 127.0.0.1:43242)
0811 01:50:29.891062 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:29.891175 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:44281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 01:50:29.891619 15999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.125:44281: Network error: Client connection negotiation failed: client connection to 127.21.214.125:44281: connect: Connection refused (error 111)
I20260811 01:50:29.982918 15999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } attempt: 3
I20260811 01:50:29.984341 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:29.983669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:44281 (local address 127.0.0.1:43244)
0811 01:50:29.983965 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:29.984112 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:44281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 01:50:29.986042 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:29.985377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:44281 (local address 127.0.0.1:43256)
0811 01:50:29.985588 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:29.985819 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:44281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 01:50:29.986268 15999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.125:44281: Network error: Client connection negotiation failed: client connection to 127.21.214.125:44281: connect: Connection refused (error 111)
I20260811 01:50:30.129540 15999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } attempt: 4
I20260811 01:50:30.131114 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.130427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:44281 (local address 127.0.0.1:43270)
0811 01:50:30.130793 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.130920 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:44281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202}
I20260811 01:50:30.132299 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.131848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:44281 (local address 127.0.0.1:43276)
0811 01:50:30.132028 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.132135 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:44281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 01:50:30.132524 15999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.125:44281: Network error: Client connection negotiation failed: client connection to 127.21.214.125:44281: connect: Connection refused (error 111)
W20260811 01:50:30.178272 16007 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 01:50:30.178678 16007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:30.178772 16007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:30.178884 16007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:30.188771 16007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:30.188887 16007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:30.188954 16007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:30.189014 16007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:30.189087 16007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:30.199720 16007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:41571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:34813,127.21.214.125:44281,127.21.214.124:45993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.125.keytab
--principal=kudu/127.21.214.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.125:44281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.214.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:30.201246 16007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:30.202704 16007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:30.216116 16013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:30.216213 16016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:30.216852 16012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:30.216946 16007 server_base.cc:1034] running on GCE node
I20260811 01:50:30.217936 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:30.219491 16007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:30.220743 16007 hybrid_clock.cc:648] HybridClock initialized: now 1786413030220680 us; error 42 us; skew 500 ppm
I20260811 01:50:30.221113 16007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:30 dist-test-slave-9gcw krb5kdc[15907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413030, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:30.224614 16007 init.cc:377] Logged in from keytab as kudu/127.21.214.125@KRBTEST.COM (short username kudu)
I20260811 01:50:30.226161 16007 webserver.cc:460] Webserver started at http://127.21.214.125:33949/ using document root <none> and password file <none>
I20260811 01:50:30.226907 16007 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:30.227049 16007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:30.227385 16007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:30.230602 16007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-1/data/instance:
uuid: "0e958b4e8c964f7daea207979e0c50e0"
format_stamp: "Formatted at 2026-08-11 01:50:30 on dist-test-slave-9gcw"
server_key: "d724830bdda85b0758426a008f59fee8"
server_key_iv: "a8a05fb0bc1faf1fa0165fbd8590c176"
server_key_version: "encryptionkey@0"
I20260811 01:50:30.231459 16007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-1/wal/instance:
uuid: "0e958b4e8c964f7daea207979e0c50e0"
format_stamp: "Formatted at 2026-08-11 01:50:30 on dist-test-slave-9gcw"
server_key: "d724830bdda85b0758426a008f59fee8"
server_key_iv: "a8a05fb0bc1faf1fa0165fbd8590c176"
server_key_version: "encryptionkey@0"
I20260811 01:50:30.236967 16007 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 01:50:30.240614 16027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:30.242029 16007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 01:50:30.242228 16007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-1/wal
uuid: "0e958b4e8c964f7daea207979e0c50e0"
format_stamp: "Formatted at 2026-08-11 01:50:30 on dist-test-slave-9gcw"
server_key: "d724830bdda85b0758426a008f59fee8"
server_key_iv: "a8a05fb0bc1faf1fa0165fbd8590c176"
server_key_version: "encryptionkey@0"
I20260811 01:50:30.242463 16007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:30.278959 16007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:30.284096 16007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:30.284386 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:30.304236 16007 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.125:44281
I20260811 01:50:30.304265 16091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.125:44281 every 8 connection(s)
I20260811 01:50:30.306063 16007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-1/data/info.pb
I20260811 01:50:30.310428 16093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:50:30.311282 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 16007
I20260811 01:50:30.311492 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-1/wal/instance
I20260811 01:50:30.312083 22361 external_mini_cluster.cc:1428] Setting key fd0ea921f782712d7268402aa573d4c2
I20260811 01:50:30.321329 16093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:50:30 dist-test-slave-9gcw krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413030, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.125@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:50:30.348121 16095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.325229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54650 (local address 127.21.214.126:34813)
0811 01:50:30.325809 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:30.325818 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:30.327079 (+ 1261us) server_negotiation.cc:407] Connection header received
0811 01:50:30.330260 (+ 3181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:30.330266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:30.330351 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:30.330445 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:30.332852 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.333681 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.336260 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.336680 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.336777 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:30.341486 (+ 4709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:30.341491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:30.341494 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:30.341543 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:30.344391 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:30.345443 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:30.345449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:30.345454 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:30.345588 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:30.346082 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:30.346086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:30.346088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:30.346463 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:30.346621 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:30.347470 (+ 849us) server_negotiation.cc:299] Negotiation successful
0811 01:50:30.347708 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":272,"threads_started":1}
WARNING: no policy specified for kudu/127.21.214.124@KRBTEST.COM; defaulting to no policy
I20260811 01:50:30.349251 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.325930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:34813 (local address 127.0.0.1:54650)
0811 01:50:30.326848 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.326903 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 01:50:30.329869 (+ 2966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:50:30.330777 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:50:30.330793 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:50:30.331659 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:50:30.332615 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:30.332644 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.333866 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.333874 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:30.336099 (+ 2225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:30.336118 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.336971 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.336984 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:30.337370 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:30.338216 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:50:30.338260 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:50:30.341220 (+ 2960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:50:30.344576 (+ 3356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:30.344586 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:30.344613 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:30.345299 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:30.345714 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:30.345719 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:30.345723 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:30.345941 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:30.346596 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:50:30.346605 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:50:30.347084 (+ 479us) client_negotiation.cc:769] Sending connection context
0811 01:50:30.347385 (+ 301us) client_negotiation.cc:240] Negotiation successful
0811 01:50:30.348000 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":403,"threads_started":1}
I20260811 01:50:30.351154 16093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.124@KRBTEST.COM" created.
Aug 11 01:50:30 dist-test-slave-9gcw krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413030, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.125@KRBTEST.COM for kudu/127.21.214.125@KRBTEST.COM
I20260811 01:50:30.370086 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.352210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:44281 (local address 127.0.0.1:43286)
0811 01:50:30.352392 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.352417 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 01:50:30.352547 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:50:30.353450 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:50:30.353454 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:50:30.353925 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:50:30.354060 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:30.354068 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.355480 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.355486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:30.356382 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:30.356394 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.357358 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.357372 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:30.357647 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:30.358380 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:50:30.358412 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:50:30.361514 (+ 3102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:50:30.366681 (+ 5167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:30.366689 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:30.366695 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:30.367213 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:30.367729 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:30.367736 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:30.367742 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:30.367914 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:30.368854 (+ 940us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:50:30.368859 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:50:30.369029 (+ 170us) client_negotiation.cc:769] Sending connection context
0811 01:50:30.369255 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 01:50:30.369601 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 01:50:30.370541 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.352203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.21.214.125:44281)
0811 01:50:30.352940 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:30.352950 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:30.352990 (+ 40us) server_negotiation.cc:407] Connection header received
0811 01:50:30.353043 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:30.353049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:30.353304 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:30.353464 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:30.354237 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.355322 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.356533 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.357121 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.357308 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:30.361781 (+ 4473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:30.361801 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:30.361825 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:30.361880 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:30.366495 (+ 4615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:30.367384 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:30.367397 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:30.367404 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:30.367582 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:30.368037 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:30.368046 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:30.368052 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:30.368705 (+ 653us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:30.368937 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:30.369428 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 01:50:30.369751 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":563,"thread_start_us":317,"threads_started":1}
I20260811 01:50:30.373309 16093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:50:30.374888 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.374230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58354)
0811 01:50:30.374459 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.374637 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 01:50:30.375294 16029 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
I20260811 01:50:30.377322 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.376709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58364)
0811 01:50:30.376939 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.377060 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 01:50:30.377575 16093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:45993: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.124.keytab.
Entry for principal kudu/127.21.214.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.124.keytab.
I20260811 01:50:30.397750 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.124:45993
--webserver_interface=127.21.214.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.124.keytab
--principal=kudu/127.21.214.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:41571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:34813,127.21.214.125:44281,127.21.214.124:45993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:50:30.427014 16093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } attempt: 1
I20260811 01:50:30.428440 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.427808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58366)
0811 01:50:30.428070 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.428220 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 01:50:30.429726 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.429233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58374)
0811 01:50:30.429437 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.429551 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 01:50:30.429975 16093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:45993: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
I20260811 01:50:30.431784 15999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } attempt: 5
Aug 11 01:50:30 dist-test-slave-9gcw krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413029, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for kudu/127.21.214.125@KRBTEST.COM
I20260811 01:50:30.445431 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.432509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43296 (local address 127.21.214.125:44281)
0811 01:50:30.432711 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:30.432718 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:30.432824 (+ 106us) server_negotiation.cc:407] Connection header received
0811 01:50:30.433021 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:30.433032 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:30.433094 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:30.433234 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:30.434235 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.434970 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.436169 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.436617 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.436763 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:30.439957 (+ 3194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:30.439962 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:30.439965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:30.439997 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:30.442738 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:30.443424 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:30.443431 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:30.443435 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:30.443562 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:30.443959 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:30.443965 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:30.443968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:30.444255 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:30.444418 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:30.444797 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 01:50:30.445031 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 01:50:30.445436 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.432488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:44281 (local address 127.0.0.1:43296)
0811 01:50:30.432694 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.432717 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:50:30.432891 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:50:30.433272 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:50:30.433276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:50:30.433823 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:50:30.434081 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:30.434092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.435113 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.435118 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:30.436005 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:30.436016 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.436771 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.436777 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:30.436991 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:30.437696 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:50:30.437714 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:50:30.439753 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:50:30.442904 (+ 3151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:30.442910 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:30.442914 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:30.443293 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:30.443681 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:30.443686 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:30.443690 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:30.443846 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:30.444385 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:50:30.444390 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:50:30.444540 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 01:50:30.444685 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 01:50:30.445018 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 01:50:30.446537 15999 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:50:30.447789 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.447293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58378)
0811 01:50:30.447588 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.447662 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197}
I20260811 01:50:30.448745 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.448417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58380)
0811 01:50:30.448559 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.448646 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 01:50:30.448975 15999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:45993: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
I20260811 01:50:30.498230 16093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } attempt: 2
I20260811 01:50:30.499614 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.499042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58390)
0811 01:50:30.499284 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.499409 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 01:50:30.500216 15999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } attempt: 1
I20260811 01:50:30.500829 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.500385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58404)
0811 01:50:30.500586 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.500673 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260811 01:50:30.501119 16093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:45993: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
I20260811 01:50:30.501302 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.500772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58418)
0811 01:50:30.500950 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.501048 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 01:50:30.502317 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.501984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58432)
0811 01:50:30.502143 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.502211 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 01:50:30.502525 15999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:45993: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
I20260811 01:50:30.570767 15999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } attempt: 2
I20260811 01:50:30.572029 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.571485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58444)
0811 01:50:30.571749 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.571862 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 01:50:30.573041 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.572692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58456)
0811 01:50:30.572874 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.572941 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 01:50:30.573298 15999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:45993: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
I20260811 01:50:30.592419 16093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } attempt: 3
I20260811 01:50:30.593794 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.593230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58466)
0811 01:50:30.593463 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.593581 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 01:50:30.595003 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.594501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58480)
0811 01:50:30.594745 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.594854 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
W20260811 01:50:30.595247 16093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:45993: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
I20260811 01:50:30.679517 15999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } attempt: 3
I20260811 01:50:30.680821 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.680350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58490)
0811 01:50:30.680560 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.680656 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 01:50:30.682468 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.682063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58504)
0811 01:50:30.682266 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.682357 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260811 01:50:30.682812 15999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:45993: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
I20260811 01:50:30.738510 16093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } attempt: 4
I20260811 01:50:30.739874 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.739203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58512)
0811 01:50:30.739504 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.739632 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 01:50:30.740927 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.740549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58516)
0811 01:50:30.740720 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.740805 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 01:50:30.741214 16093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:45993: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
W20260811 01:50:30.771211 16101 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 01:50:30.771543 16101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:30.771682 16101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:30.771821 16101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:30.779522 16101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:30.779627 16101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:30.779728 16101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:30.779829 16101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:30.779945 16101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:30.790252 16101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:41571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:34813,127.21.214.125:44281,127.21.214.124:45993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.124.keytab
--principal=kudu/127.21.214.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.124:45993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.214.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:30.791693 16101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:30.793128 16101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:30.806006 16107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:30.806083 16111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:30.806602 16106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:30.807847 16101 server_base.cc:1034] running on GCE node
I20260811 01:50:30.808691 16101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:30.810371 16101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:30.811676 16101 hybrid_clock.cc:648] HybridClock initialized: now 1786413030811575 us; error 42 us; skew 500 ppm
I20260811 01:50:30.812064 16101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:30 dist-test-slave-9gcw krb5kdc[15907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413030, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:30.815544 16101 init.cc:377] Logged in from keytab as kudu/127.21.214.124@KRBTEST.COM (short username kudu)
I20260811 01:50:30.817070 16101 webserver.cc:460] Webserver started at http://127.21.214.124:46417/ using document root <none> and password file <none>
I20260811 01:50:30.817838 16101 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:30.817978 16101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:30.818321 16101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:30.821570 16101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-2/data/instance:
uuid: "c0c28c3c8bd741bb87765dd763eb09d4"
format_stamp: "Formatted at 2026-08-11 01:50:30 on dist-test-slave-9gcw"
server_key: "07cc50ac49e989d461fea7c756fb6379"
server_key_iv: "caf8ab231429ba41e976d3716f504697"
server_key_version: "encryptionkey@0"
I20260811 01:50:30.822444 16101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-2/wal/instance:
uuid: "c0c28c3c8bd741bb87765dd763eb09d4"
format_stamp: "Formatted at 2026-08-11 01:50:30 on dist-test-slave-9gcw"
server_key: "07cc50ac49e989d461fea7c756fb6379"
server_key_iv: "caf8ab231429ba41e976d3716f504697"
server_key_version: "encryptionkey@0"
I20260811 01:50:30.827999 16101 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 01:50:30.831631 16118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:30.833474 16101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 01:50:30.833706 16101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-2/wal
uuid: "c0c28c3c8bd741bb87765dd763eb09d4"
format_stamp: "Formatted at 2026-08-11 01:50:30 on dist-test-slave-9gcw"
server_key: "07cc50ac49e989d461fea7c756fb6379"
server_key_iv: "caf8ab231429ba41e976d3716f504697"
server_key_version: "encryptionkey@0"
I20260811 01:50:30.833922 16101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:30.852553 16101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:30.857380 16101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:30.857645 16101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:30.860044 15999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } attempt: 4
I20260811 01:50:30.861305 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.860810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58520)
0811 01:50:30.861013 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.861099 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 01:50:30.862433 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.862097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58532)
0811 01:50:30.862247 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.862315 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 01:50:30.862730 15999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:45993: Network error: Client connection negotiation failed: client connection to 127.21.214.124:45993: connect: Connection refused (error 111)
I20260811 01:50:30.879752 16101 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.124:45993
I20260811 01:50:30.879797 16178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.124:45993 every 8 connection(s)
I20260811 01:50:30.881487 16101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-2/data/info.pb
I20260811 01:50:30.884114 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 16101
I20260811 01:50:30.884337 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786412636131983-22361-0/minicluster-data/master-2/wal/instance
I20260811 01:50:30.884851 22361 external_mini_cluster.cc:1428] Setting key 2de67a8663c3a3fe4bd48ded7cd14953
I20260811 01:50:30.885820 16179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:50:30.896222 16179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:50:30 dist-test-slave-9gcw krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.124@KRBTEST.COM
Aug 11 01:50:30 dist-test-slave-9gcw krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413030, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.124@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:50:30.911950 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.887156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58546 (local address 127.21.214.124:45993)
0811 01:50:30.887997 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:30.888013 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:30.888065 (+ 52us) server_negotiation.cc:407] Connection header received
0811 01:50:30.890734 (+ 2669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:30.890769 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:30.891351 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:30.891858 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:30.893471 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.895514 (+ 2043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.896911 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.897981 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.898167 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:30.903622 (+ 5455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:30.903629 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:30.903641 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:30.903677 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:30.907083 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:30.907933 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:30.907947 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:30.907954 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:30.908189 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:30.908646 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:30.908653 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:30.908659 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:30.909394 (+ 735us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:30.909730 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:30.910225 (+ 495us) server_negotiation.cc:299] Negotiation successful
0811 01:50:30.910859 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"spinlock_wait_cycles":19712,"thread_start_us":356,"threads_started":1}
I20260811 01:50:30.916096 16183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.899391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:34813 (local address 127.0.0.1:54666)
0811 01:50:30.900329 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.900399 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 01:50:30.900599 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:50:30.901397 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:50:30.901402 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:50:30.902315 (+ 913us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:50:30.902677 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:30.902688 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.904070 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.904075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:30.905379 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:30.905393 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.906188 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.906196 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:30.906376 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:30.906997 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:50:30.907014 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:50:30.909475 (+ 2461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:50:30.912804 (+ 3329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:30.912816 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:30.912824 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:30.913411 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:30.913883 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:30.913888 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:30.913893 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:30.914050 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:30.914713 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:50:30.914721 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:50:30.914982 (+ 261us) client_negotiation.cc:769] Sending connection context
0811 01:50:30.915169 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 01:50:30.915546 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":358,"threads_started":1}
I20260811 01:50:30.916177 16184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.899451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54666 (local address 127.21.214.126:34813)
0811 01:50:30.900983 (+ 1532us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:30.900989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:30.901010 (+ 21us) server_negotiation.cc:407] Connection header received
0811 01:50:30.901076 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:30.901081 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:30.901212 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:30.901341 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:30.902867 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.903892 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.905546 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.905965 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.906109 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:30.909688 (+ 3579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:30.909695 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:30.909698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:30.909733 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:30.912655 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:30.913550 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:30.913556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:30.913560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:30.913718 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:30.914165 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:30.914168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:30.914170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:30.914566 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:30.914722 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:30.915400 (+ 678us) server_negotiation.cc:299] Negotiation successful
0811 01:50:30.915649 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1397,"thread_start_us":1226,"threads_started":1}
I20260811 01:50:30.917981 16179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:50:30 dist-test-slave-9gcw krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
Aug 11 01:50:30 dist-test-slave-9gcw krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413030, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.124@KRBTEST.COM for kudu/127.21.214.125@KRBTEST.COM
I20260811 01:50:30.935184 16185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.915526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54680 (local address 127.21.214.126:34813)
0811 01:50:30.916438 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:30.916443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:30.916473 (+ 30us) server_negotiation.cc:407] Connection header received
0811 01:50:30.916513 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:30.916516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:30.916601 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:30.916725 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:30.918517 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.919695 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.921123 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.921624 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.921720 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:30.928421 (+ 6701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:30.928427 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:30.928430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:30.928490 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:30.932081 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:30.932996 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:30.933001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:30.933003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:30.933137 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:30.933595 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:30.933601 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:30.933606 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:30.933982 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:30.934257 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:30.934572 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 01:50:30.934756 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":761,"thread_start_us":514,"threads_started":1}
I20260811 01:50:30.935485 16183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.918825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:44281 (local address 127.0.0.1:43300)
0811 01:50:30.918985 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.919005 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 01:50:30.919095 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:50:30.919474 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:50:30.919477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:50:30.919865 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:50:30.920050 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:30.920060 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.921030 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.921038 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:30.922277 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:30.922290 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.923252 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.923259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:30.923595 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:30.924246 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:50:30.924272 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:50:30.927961 (+ 3689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:50:30.932610 (+ 4649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:30.932616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:30.932619 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:30.933206 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:30.933669 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:30.933673 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:30.933675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:30.933817 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:30.934454 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:50:30.934459 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:50:30.934639 (+ 180us) client_negotiation.cc:769] Sending connection context
0811 01:50:30.934841 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 01:50:30.935129 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 01:50:30.935807 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.918850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43300 (local address 127.21.214.125:44281)
0811 01:50:30.919034 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:30.919040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:30.919059 (+ 19us) server_negotiation.cc:407] Connection header received
0811 01:50:30.919214 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:30.919222 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:30.919318 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:30.919449 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:30.920202 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.920879 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.922419 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.923065 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.923210 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:30.928209 (+ 4999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:30.928217 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:30.928224 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:30.928268 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:30.932454 (+ 4186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:30.933348 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:30.933354 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:30.933358 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:30.933513 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:30.933947 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:30.933952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:30.933956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:30.934314 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:30.934472 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:30.934977 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 01:50:30.935215 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 01:50:30.937068 16179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:50:30 dist-test-slave-9gcw krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413030, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.124@KRBTEST.COM for kudu/127.21.214.124@KRBTEST.COM
Aug 11 01:50:30 dist-test-slave-9gcw krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.125@KRBTEST.COM
I20260811 01:50:30.950757 16183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.937885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58558)
0811 01:50:30.938142 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:30.938154 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 01:50:30.938272 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:50:30.938559 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:50:30.938563 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:50:30.938959 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:50:30.939158 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:30.939170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.940074 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.940081 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:30.940937 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:30.940948 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.941706 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.941712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:30.941946 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:30.942581 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:50:30.942605 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:50:30.944640 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:50:30.948111 (+ 3471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:30.948119 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:30.948126 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:30.948607 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:30.949046 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:30.949050 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:30.949053 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:30.949293 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:30.949955 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:50:30.949958 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:50:30.950078 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 01:50:30.950205 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 01:50:30.950419 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187}
I20260811 01:50:30.951057 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.937866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58558 (local address 127.21.214.124:45993)
0811 01:50:30.938051 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:30.938057 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:30.938220 (+ 163us) server_negotiation.cc:407] Connection header received
0811 01:50:30.938387 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:30.938394 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:30.938448 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:30.938543 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:30.939284 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.939950 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.941076 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.941546 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.941690 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:30.944853 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:30.944860 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:30.944866 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:30.944900 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:30.947975 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:30.948743 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:30.948751 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:30.948759 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:30.948904 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:30.949417 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:30.949422 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:30.949427 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:30.949805 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:30.949964 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:30.950385 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 01:50:30.950648 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 01:50:30.953630 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:30.937229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43302 (local address 127.21.214.125:44281)
0811 01:50:30.937424 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:30.937430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:30.937446 (+ 16us) server_negotiation.cc:407] Connection header received
0811 01:50:30.937588 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:30.937593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:30.937688 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:30.938039 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:30.939513 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.940212 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.941532 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:30.941950 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:30.942089 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:30.947175 (+ 5086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:30.947181 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:30.947184 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:30.947222 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:30.950602 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:30.951532 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:30.951536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:30.951538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:30.951741 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:30.952171 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:30.952179 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:30.952183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:30.952504 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:30.952771 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:30.953018 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 01:50:30.953298 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 01:50:30.957000 16179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4: Bootstrap starting.
I20260811 01:50:30.958734 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:50:30.960543 16179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:30.961784 16179 log.cc:824] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4: Log is configured to *not* fsync() on all Append() calls
I20260811 01:50:30.961495 15973 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54680
I20260811 01:50:30.964807 16179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4: No bootstrap required, opened a new log
I20260811 01:50:30.972083 16179 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } } peers { permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } } peers { permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } }
I20260811 01:50:30.972486 16179 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0c28c3c8bd741bb87765dd763eb09d4, State: Initialized, Role: FOLLOWER
I20260811 01:50:30.972947 16179 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } } peers { permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } } peers { permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } }
I20260811 01:50:30.973903 16187 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } } peers { permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } } peers { permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } } }
I20260811 01:50:30.974238 16187 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:50:30.975121 16179 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 01:50:30.985232 16198 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 01:50:30.985394 16198 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 01:50:30.983026 16101 master_runner.cc:186] Error getting master registration for 127.21.214.126:34813: OK, instance_id { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" instance_seqno: 1786413029717516 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 01:50:30.994621 16101 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:50:31.040462 16093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } attempt: 5
Aug 11 01:50:31 dist-test-slave-9gcw krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413030, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.125@KRBTEST.COM for kudu/127.21.214.124@KRBTEST.COM
I20260811 01:50:31.063294 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:31.041498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58566 (local address 127.21.214.124:45993)
0811 01:50:31.041732 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:31.041741 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:31.041764 (+ 23us) server_negotiation.cc:407] Connection header received
0811 01:50:31.041835 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:31.041843 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:31.041967 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:31.042103 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:31.042958 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:31.043819 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:31.045775 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:31.046631 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:31.046784 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:31.055375 (+ 8591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:31.055385 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:31.055391 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:31.055439 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:31.059723 (+ 4284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:31.060778 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:31.060784 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:31.060788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:31.060947 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:31.061469 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:31.061474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:31.061478 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:31.061916 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:31.062085 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:31.062610 (+ 525us) server_negotiation.cc:299] Negotiation successful
0811 01:50:31.062897 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 01:50:31.063388 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:31.041294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58566)
0811 01:50:31.041523 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:31.041548 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 01:50:31.041695 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:50:31.042149 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:50:31.042156 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:50:31.042610 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:50:31.042775 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:31.042785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:31.044037 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:31.044042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:31.045555 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:31.045575 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:31.047093 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:31.047102 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:31.047404 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:31.048179 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:50:31.048198 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:50:31.055041 (+ 6843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:50:31.059931 (+ 4890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:31.059941 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:31.059948 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:31.060597 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:31.061062 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:31.061070 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:31.061075 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:31.061326 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:31.062043 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:50:31.062051 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:50:31.062272 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 01:50:31.062454 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 01:50:31.062856 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 01:50:31.070676 16093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0: Bootstrap starting.
I20260811 01:50:31.075932 16093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:31.077766 16093 log.cc:824] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0: Log is configured to *not* fsync() on all Append() calls
I20260811 01:50:31.082090 16093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0: No bootstrap required, opened a new log
I20260811 01:50:31.096560 16093 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } } peers { permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } } peers { permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } }
I20260811 01:50:31.097158 16093 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e958b4e8c964f7daea207979e0c50e0, State: Initialized, Role: FOLLOWER
I20260811 01:50:31.097891 16093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } } peers { permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } } peers { permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } }
I20260811 01:50:31.099562 16201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } } peers { permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } } peers { permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } } }
I20260811 01:50:31.100366 16201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:50:31.101037 16093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 01:50:31.109887 16124 tablet.cc:2369] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:50:31.140873 15999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } attempt: 5
Aug 11 01:50:31 dist-test-slave-9gcw krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413029, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for kudu/127.21.214.124@KRBTEST.COM
I20260811 01:50:31.140693 16007 master_runner.cc:186] Error getting master registration for 127.21.214.126:34813: OK, instance_id { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" instance_seqno: 1786413029717516 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 01:50:31.152603 16007 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 01:50:31.164203 16213 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 01:50:31.164361 16213 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 01:50:31.165853 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:31.141590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:45993 (local address 127.0.0.1:58572)
0811 01:50:31.141804 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:31.141828 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 01:50:31.141955 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:50:31.142913 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:50:31.142920 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:50:31.143556 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:50:31.143782 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:31.143796 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:31.145386 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:31.145392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:31.146964 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:31.146982 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:31.148328 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:31.148335 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:31.148590 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:31.149369 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:50:31.149422 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:50:31.152605 (+ 3183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:50:31.161094 (+ 8489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:31.161103 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:31.161110 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:31.161873 (+ 763us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:31.162862 (+ 989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:31.162871 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:31.162877 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:31.163125 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:31.164389 (+ 1264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:50:31.164406 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:50:31.164639 (+ 233us) client_negotiation.cc:769] Sending connection context
0811 01:50:31.164800 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 01:50:31.165185 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 01:50:31.168195 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:31.142231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58572 (local address 127.21.214.124:45993)
0811 01:50:31.142428 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:31.142434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:31.142454 (+ 20us) server_negotiation.cc:407] Connection header received
0811 01:50:31.142524 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:31.142530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:31.142608 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:31.142719 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:31.143983 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:31.145209 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:31.147166 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:31.147934 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:31.148051 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:31.153018 (+ 4967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:31.153056 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:31.153066 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:31.153184 (+ 118us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:31.160780 (+ 7596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:31.162230 (+ 1450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:31.162240 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:31.162246 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:31.162593 (+ 347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:31.163394 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:31.163402 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:31.163407 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:31.164184 (+ 777us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:31.166508 (+ 2324us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:31.166833 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 01:50:31.167359 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 01:50:31.175659 15999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4: Bootstrap starting.
I20260811 01:50:31.183580 15999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:31.186177 15999 log.cc:824] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4: Log is configured to *not* fsync() on all Append() calls
I20260811 01:50:31.204483 15999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4: No bootstrap required, opened a new log
I20260811 01:50:31.213390 15999 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } } peers { permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } } peers { permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } }
I20260811 01:50:31.214064 15999 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17aceda2affe4b5dad8c4c54dc5a6ff4, State: Initialized, Role: FOLLOWER
I20260811 01:50:31.214761 15999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } } peers { permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } } peers { permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } }
I20260811 01:50:31.222100 16216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } } peers { permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } } peers { permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } } }
I20260811 01:50:31.224759 15973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 01:50:31.224685 15999 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:50:31.224957 15973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:50:31.230787 15973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } } peers { permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } } peers { permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } }
I20260811 01:50:31.233563 16066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0e958b4e8c964f7daea207979e0c50e0"
I20260811 01:50:31.234230 16066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:50:31.238958 16066 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17aceda2affe4b5dad8c4c54dc5a6ff4 in term 1.
I20260811 01:50:31.239981 15944 leader_election.cc:302] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e958b4e8c964f7daea207979e0c50e0, 17aceda2affe4b5dad8c4c54dc5a6ff4; no voters:
I20260811 01:50:31.243752 16150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c0c28c3c8bd741bb87765dd763eb09d4"
I20260811 01:50:31.244438 16150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:50:31.249365 16150 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17aceda2affe4b5dad8c4c54dc5a6ff4 in term 1.
I20260811 01:50:31.250566 15973 leader_election.cc:288] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [CANDIDATE]: Term 1 election: Requested vote from peers 0e958b4e8c964f7daea207979e0c50e0 (127.21.214.125:44281), c0c28c3c8bd741bb87765dd763eb09d4 (127.21.214.124:45993)
I20260811 01:50:31.224833 16216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:50:31.266609 16216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:50:31.266993 16216 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [term 1 LEADER]: Becoming Leader. State: Replica: 17aceda2affe4b5dad8c4c54dc5a6ff4, State: Running, Role: LEADER
I20260811 01:50:31.267777 16216 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } } peers { permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } } peers { permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } }
W20260811 01:50:31.287441 16035 tablet.cc:2369] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:50:31.288570 16218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17aceda2affe4b5dad8c4c54dc5a6ff4. Latest consensus state: current_term: 1 leader_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } } peers { permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } } peers { permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } } }
I20260811 01:50:31.289276 16218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:31.300673 16224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 01:50:31.320977 16224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:50:31.415834 16150 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4 [term 1 FOLLOWER]: Refusing update from remote peer 17aceda2affe4b5dad8c4c54dc5a6ff4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:50:31.418397 16066 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0 [term 1 FOLLOWER]: Refusing update from remote peer 17aceda2affe4b5dad8c4c54dc5a6ff4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:50:31.419705 16216 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:50:31.421492 16216 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:50:31.449201 16185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:31.311226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54690 (local address 127.21.214.126:34813)
0811 01:50:31.311427 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:31.311434 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:31.339231 (+ 27797us) server_negotiation.cc:407] Connection header received
0811 01:50:31.339614 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:31.339621 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:31.339706 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:31.339798 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:31.369788 (+ 29990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:31.371123 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:31.387044 (+ 15921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:31.387958 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:31.388098 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:31.440203 (+ 52105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:31.440214 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:31.440222 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:31.440299 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:31.445084 (+ 4785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:31.446507 (+ 1423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:31.446515 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:31.446519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:31.446754 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:31.447245 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:31.447262 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:31.447265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:31.447687 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:31.447926 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:31.448569 (+ 643us) server_negotiation.cc:299] Negotiation successful
0811 01:50:31.448821 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 01:50:31.453322 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:31.330372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43316 (local address 127.21.214.125:44281)
0811 01:50:31.330581 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:31.330591 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:31.354391 (+ 23800us) server_negotiation.cc:407] Connection header received
0811 01:50:31.361360 (+ 6969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:31.361369 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:31.361492 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:31.361574 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:31.379209 (+ 17635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:31.380582 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:31.393393 (+ 12811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:31.394368 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:31.394517 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:31.422551 (+ 28034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:31.422558 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:31.422562 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:31.422608 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:31.426886 (+ 4278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:31.434037 (+ 7151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:31.434045 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:31.434051 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:31.434280 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:31.435594 (+ 1314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:31.435602 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:31.435608 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:31.436689 (+ 1081us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:31.436974 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:31.452112 (+ 15138us) server_negotiation.cc:299] Negotiation successful
0811 01:50:31.452512 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 01:50:31.454864 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:31.324049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58588 (local address 127.21.214.124:45993)
0811 01:50:31.324298 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:31.324307 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:31.354859 (+ 30552us) server_negotiation.cc:407] Connection header received
0811 01:50:31.357703 (+ 2844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:31.357713 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:31.357828 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:31.357937 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:31.377383 (+ 19446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:31.378808 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:31.395050 (+ 16242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:31.396127 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:31.396282 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:31.427752 (+ 31470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:31.427772 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:31.427777 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:31.427826 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:31.432238 (+ 4412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:31.434837 (+ 2599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:31.434846 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:31.434852 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:31.435044 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:31.437620 (+ 2576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:31.437629 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:31.437635 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:31.438171 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:31.438514 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:31.454133 (+ 15619us) server_negotiation.cc:299] Negotiation successful
0811 01:50:31.454450 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 01:50:31.456755 15948 tablet.cc:2369] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:50:31.458058 16187 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17aceda2affe4b5dad8c4c54dc5a6ff4. Latest consensus state: current_term: 1 leader_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } } peers { permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } } peers { permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } } }
I20260811 01:50:31.458385 16187 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:50:31.466688 16201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17aceda2affe4b5dad8c4c54dc5a6ff4. Latest consensus state: current_term: 1 leader_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } } peers { permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } } peers { permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } } }
I20260811 01:50:31.467156 16201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:50:31.470968 16245 mvcc.cc:205] Tried to move back new op lower bound from 7317147776664567808 to 7317147776089669632. Current Snapshot: MvccSnapshot[applied={T|T < 7317147776664567808}]
I20260811 01:50:31.477367 16247 mvcc.cc:205] Tried to move back new op lower bound from 7317147776664567808 to 7317147776089669632. Current Snapshot: MvccSnapshot[applied={T|T < 7317147776664567808}]
I20260811 01:50:31.488190 16218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } } peers { permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } } peers { permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } } }
I20260811 01:50:31.488559 16218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:31.488996 16218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } } peers { permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } } peers { permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } } }
I20260811 01:50:31.489274 16218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:31.492695 16224 catalog_manager.cc:1347] Generated new cluster ID: d8a6259a60b04a8d8342caa9ed103cdb
I20260811 01:50:31.492805 16224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:50:31.494174 16187 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } } peers { permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } } peers { permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } } }
I20260811 01:50:31.494294 16201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aceda2affe4b5dad8c4c54dc5a6ff4" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 34813 } } peers { permanent_uuid: "0e958b4e8c964f7daea207979e0c50e0" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 44281 } } peers { permanent_uuid: "c0c28c3c8bd741bb87765dd763eb09d4" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 45993 } } }
I20260811 01:50:31.494498 16187 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:50:31.494566 16201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:50:31.570518 16224 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:50:31.572719 16224 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:50:31.609064 16224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17aceda2affe4b5dad8c4c54dc5a6ff4: Generated new TSK 0
I20260811 01:50:31.610363 16224 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 01:50:31.735734 16185 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:31.722701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54696 (local address 127.21.214.126:34813)
0811 01:50:31.723190 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:31.723200 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:31.731804 (+ 8604us) server_negotiation.cc:407] Connection header received
0811 01:50:31.733069 (+ 1265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:31.733077 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:31.733203 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:31.733292 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:31.735426 (+ 2134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54696: BlockingRecv error: recv got EOF from 127.0.0.1:54696 (error 108)
Metrics: {"server-negotiator.queue_time_us":305}
W20260811 01:50:31.740404 16182 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:31.722721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58600 (local address 127.21.214.124:45993)
0811 01:50:31.722980 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:31.722991 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:31.737449 (+ 14458us) server_negotiation.cc:407] Connection header received
0811 01:50:31.738516 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:31.738523 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:31.738618 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:31.738726 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:31.740135 (+ 1409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58600: BlockingRecv error: recv got EOF from 127.0.0.1:58600 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 01:50:31.747370 16098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:31.733846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43320 (local address 127.21.214.125:44281)
0811 01:50:31.734179 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:31.734187 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:31.741945 (+ 7758us) server_negotiation.cc:407] Connection header received
0811 01:50:31.745420 (+ 3475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:31.745428 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:31.745539 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:31.745626 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:31.747073 (+ 1447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43320: BlockingRecv error: recv got EOF from 127.0.0.1:43320 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 01:50:31.776281 16098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:31.773562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43328 (local address 127.21.214.125:44281)
0811 01:50:31.773854 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:31.773864 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:31.774440 (+ 576us) server_negotiation.cc:407] Connection header received
0811 01:50:31.774746 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:31.774750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:31.774870 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:31.775007 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:31.775913 (+ 906us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43328: BlockingRecv error: recv got EOF from 127.0.0.1:43328 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260811 01:50:31.787672 16098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:31.781128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43342 (local address 127.21.214.125:44281)
0811 01:50:31.781419 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:31.781429 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:31.785968 (+ 4539us) server_negotiation.cc:407] Connection header received
0811 01:50:31.786245 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:31.786252 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:31.786344 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:31.786489 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:31.787407 (+ 918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43342: BlockingRecv error: recv got EOF from 127.0.0.1:43342 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 01:50:31.795480 16098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:31.793027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43350 (local address 127.21.214.125:44281)
0811 01:50:31.793299 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:31.793310 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:31.793909 (+ 599us) server_negotiation.cc:407] Connection header received
0811 01:50:31.794149 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:31.794156 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:31.794272 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:31.794393 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:31.795165 (+ 772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43350: BlockingRecv error: recv got EOF from 127.0.0.1:43350 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 01:50:31.806169 16098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:31.803576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43354 (local address 127.21.214.125:44281)
0811 01:50:31.803855 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:31.803865 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:31.804443 (+ 578us) server_negotiation.cc:407] Connection header received
0811 01:50:31.804692 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:31.804699 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:31.804794 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:31.804919 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:31.805839 (+ 920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43354: BlockingRecv error: recv got EOF from 127.0.0.1:43354 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 01:50:31.820168 16098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:31.817695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43360 (local address 127.21.214.125:44281)
0811 01:50:31.817980 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:31.817990 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:31.818575 (+ 585us) server_negotiation.cc:407] Connection header received
0811 01:50:31.818833 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:31.818843 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:31.818924 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:31.819046 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:31.819901 (+ 855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43360: BlockingRecv error: recv got EOF from 127.0.0.1:43360 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260811 01:50:31.842790 16098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:31.840136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43368 (local address 127.21.214.125:44281)
0811 01:50:31.840428 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:31.840439 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:31.841077 (+ 638us) server_negotiation.cc:407] Connection header received
0811 01:50:31.841379 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:31.841384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:31.841487 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:31.841612 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:31.842467 (+ 855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43368: BlockingRecv error: recv got EOF from 127.0.0.1:43368 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260811 01:50:31.881379 16098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:31.878772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43376 (local address 127.21.214.125:44281)
0811 01:50:31.879045 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:31.879055 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:31.879778 (+ 723us) server_negotiation.cc:407] Connection header received
0811 01:50:31.880023 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:31.880029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:31.880120 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:31.880237 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:31.881079 (+ 842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43376: BlockingRecv error: recv got EOF from 127.0.0.1:43376 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 01:50:31.951980 16098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:31.949608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43380 (local address 127.21.214.125:44281)
0811 01:50:31.949963 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:31.949971 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:31.950433 (+ 462us) server_negotiation.cc:407] Connection header received
0811 01:50:31.950698 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:31.950703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:31.950778 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:31.950927 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:31.951746 (+ 819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43380: BlockingRecv error: recv got EOF from 127.0.0.1:43380 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
I20260811 01:50:31.987438 16198 catalog_manager.cc:1259] Loaded cluster ID: d8a6259a60b04a8d8342caa9ed103cdb
I20260811 01:50:31.987551 16198 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4: loading cluster ID for follower catalog manager: success
I20260811 01:50:31.990272 16198 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4: acquiring CA information for follower catalog manager: success
I20260811 01:50:31.992415 16198 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c0c28c3c8bd741bb87765dd763eb09d4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 01:50:32.096115 16098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:32.094189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43388 (local address 127.21.214.125:44281)
0811 01:50:32.094401 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:32.094406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:32.094648 (+ 242us) server_negotiation.cc:407] Connection header received
0811 01:50:32.094906 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:32.094911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:32.094988 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:32.095077 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:32.095884 (+ 807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43388: BlockingRecv error: recv got EOF from 127.0.0.1:43388 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 01:50:32.166687 16213 catalog_manager.cc:1259] Loaded cluster ID: d8a6259a60b04a8d8342caa9ed103cdb
I20260811 01:50:32.166849 16213 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0: loading cluster ID for follower catalog manager: success
I20260811 01:50:32.169773 16213 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0: acquiring CA information for follower catalog manager: success
I20260811 01:50:32.171657 16213 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0e958b4e8c964f7daea207979e0c50e0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:50:32.365684 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:32.359200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52408 (local address 127.21.214.125:44281)
0811 01:50:32.359400 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:32.359406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:32.359695 (+ 289us) server_negotiation.cc:407] Connection header received
0811 01:50:32.359878 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:32.359883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:32.360062 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:32.360193 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:50:32.360819 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:32.361693 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:32.363267 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:32.363670 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:32.363768 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:32.364360 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:50:32.364944 (+ 584us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:50:32.365064 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:32.365354 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 01:50:32.365436 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 01:50:32.379540 16330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:32.372754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35342 (local address 127.21.214.124:45993)
0811 01:50:32.373330 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:32.373335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:32.373356 (+ 21us) server_negotiation.cc:407] Connection header received
0811 01:50:32.373664 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:32.373670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:32.373843 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:32.374023 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:50:32.374434 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:32.375374 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:32.376932 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:32.377475 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:32.377639 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:32.378230 (+ 591us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:50:32.378876 (+ 646us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:50:32.379063 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:32.379118 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 01:50:32.379207 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":270,"threads_started":1}
I20260811 01:50:32.398316 16330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:32.386565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35352 (local address 127.21.214.124:45993)
0811 01:50:32.387079 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:32.387089 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:32.391815 (+ 4726us) server_negotiation.cc:407] Connection header received
0811 01:50:32.392022 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:32.392030 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:32.392160 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:32.392302 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:50:32.393266 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:32.394378 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:32.396118 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:32.396778 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:32.396912 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:32.397497 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:50:32.397631 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:50:32.397721 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:32.397913 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 01:50:32.398004 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345}
I20260811 01:50:32.399542 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:32.386379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52416 (local address 127.21.214.125:44281)
0811 01:50:32.386581 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:32.386588 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:32.393080 (+ 6492us) server_negotiation.cc:407] Connection header received
0811 01:50:32.393263 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:32.393268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:32.393372 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:32.393470 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:50:32.394039 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:32.395208 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:32.397088 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:32.397755 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:32.397901 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:32.398673 (+ 772us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:50:32.398829 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:50:32.398901 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:32.399155 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 01:50:32.399259 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 01:50:32.400435 16336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:32.386290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46190 (local address 127.21.214.126:34813)
0811 01:50:32.392573 (+ 6283us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:32.392581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:32.392606 (+ 25us) server_negotiation.cc:407] Connection header received
0811 01:50:32.392744 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:32.392752 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:32.392998 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:32.393241 (+ 243us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:50:32.394298 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:32.395438 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:32.397536 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:32.398166 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:32.398291 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:32.398987 (+ 696us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:50:32.399446 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:50:32.399579 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:32.399997 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 01:50:32.400094 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6140,"thread_start_us":6036,"threads_started":1}
I20260811 01:50:32.405966 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 15923
I20260811 01:50:32.407107 15994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:50:32.521920 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 15923
W20260811 01:50:32.549697 16030 connection.cc:535] server connection from 127.0.0.1:43296 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 01:50:32.550129 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 16007
I20260811 01:50:32.551281 16087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:50:32.664634 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 16007
I20260811 01:50:32.692124 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 16101
I20260811 01:50:32.693393 16174 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:50:32.802704 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 16101
2026-08-11T01:50:32Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5292 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 01:50:32 dist-test-slave-9gcw krb5kdc[16348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 01:50:32 dist-test-slave-9gcw krb5kdc[16348](info): set up 2 sockets
Aug 11 01:50:32 dist-test-slave-9gcw krb5kdc[16348](info): commencing operation
krb5kdc: starting...
W20260811 01:50:34.059872 22361 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.007s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/test-user.keytab.
Aug 11 01:50:34 dist-test-slave-9gcw krb5kdc[16348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T01:50:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T01:50:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.214.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
Entry for principal kudu/127.21.214.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab.
I20260811 01:50:34.429349 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:44133
--webserver_interface=127.21.214.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:39019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:44133,127.21.214.125:38023,127.21.214.124:41081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 01:50:34.799577 16365 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 01:50:34.800002 16365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:34.800143 16365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:34.800279 16365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:34.807876 16365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:34.807991 16365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:34.808096 16365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:34.808182 16365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:34.808295 16365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:34.818883 16365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:39019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:44133,127.21.214.125:38023,127.21.214.124:41081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.126.keytab
--principal=kudu/127.21.214.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.126:44133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.214.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:34.820395 16365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:34.821936 16365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:34.834910 16373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:34.834882 16375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:34.835167 16371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:34.836400 16365 server_base.cc:1034] running on GCE node
I20260811 01:50:34.837337 16365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:34.838940 16365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:34.840211 16365 hybrid_clock.cc:648] HybridClock initialized: now 1786413034840134 us; error 54 us; skew 500 ppm
I20260811 01:50:34.840577 16365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:34 dist-test-slave-9gcw krb5kdc[16348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413034, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:34.848989 16365 init.cc:377] Logged in from keytab as kudu/127.21.214.126@KRBTEST.COM (short username kudu)
I20260811 01:50:34.850817 16365 webserver.cc:460] Webserver started at http://127.21.214.126:46357/ using document root <none> and password file <none>
I20260811 01:50:34.851660 16365 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:34.851816 16365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:34.852202 16365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:34.855615 16365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-0/data/instance:
uuid: "6e3f9cf0caa94cee9abeef78d2b61425"
format_stamp: "Formatted at 2026-08-11 01:50:34 on dist-test-slave-9gcw"
server_key: "75a2c29effa48b32b23cbcf770137164"
server_key_iv: "0ca578d32ca3e820406b383bc4c6db90"
server_key_version: "encryptionkey@0"
I20260811 01:50:34.856565 16365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-0/wal/instance:
uuid: "6e3f9cf0caa94cee9abeef78d2b61425"
format_stamp: "Formatted at 2026-08-11 01:50:34 on dist-test-slave-9gcw"
server_key: "75a2c29effa48b32b23cbcf770137164"
server_key_iv: "0ca578d32ca3e820406b383bc4c6db90"
server_key_version: "encryptionkey@0"
I20260811 01:50:34.862391 16365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 01:50:34.866344 16384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:34.867874 16365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 01:50:34.868101 16365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-0/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-0/wal
uuid: "6e3f9cf0caa94cee9abeef78d2b61425"
format_stamp: "Formatted at 2026-08-11 01:50:34 on dist-test-slave-9gcw"
server_key: "75a2c29effa48b32b23cbcf770137164"
server_key_iv: "0ca578d32ca3e820406b383bc4c6db90"
server_key_version: "encryptionkey@0"
I20260811 01:50:34.868360 16365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:34.894321 16365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:34.899334 16365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:34.899686 16365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:34.922276 16365 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.126:44133
I20260811 01:50:34.922322 16441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.126:44133 every 8 connection(s)
I20260811 01:50:34.924853 16365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-0/data/info.pb
I20260811 01:50:34.927271 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 16365
I20260811 01:50:34.927495 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-0/wal/instance
I20260811 01:50:34.928131 22361 external_mini_cluster.cc:1428] Setting key 5f88e8b4d58ea118981696dd5a395b4e
I20260811 01:50:34.930622 16442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:50:34.938221 16442 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:50:34 dist-test-slave-9gcw krb5kdc[16348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413034, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.214.125@KRBTEST.COM; defaulting to no policy
I20260811 01:50:34.963666 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:34.941834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:44133 (local address 127.0.0.1:60034)
0811 01:50:34.943073 (+ 1239us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:34.943120 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 01:50:34.945775 (+ 2655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:50:34.946710 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:50:34.946716 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:50:34.947504 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:50:34.947867 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:34.947886 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:34.949639 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:34.949647 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:34.950816 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:34.950828 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:34.951827 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:34.951834 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:34.952020 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:34.952729 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:50:34.952760 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:50:34.955162 (+ 2402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:50:34.959100 (+ 3938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:34.959112 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:34.959125 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:34.959539 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:34.959974 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:34.959979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:34.959983 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:34.960148 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:34.960948 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:50:34.960956 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:50:34.961224 (+ 268us) client_negotiation.cc:769] Sending connection context
0811 01:50:34.961502 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 01:50:34.962175 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1020,"mutex_wait_us":153,"spinlock_wait_cycles":39680,"thread_start_us":788,"threads_started":1}
I20260811 01:50:34.963685 16445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:34.941833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60034 (local address 127.21.214.126:44133)
0811 01:50:34.942779 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:34.942801 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:34.943262 (+ 461us) server_negotiation.cc:407] Connection header received
0811 01:50:34.946190 (+ 2928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:34.946196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:34.946469 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:34.946697 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:34.948091 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:34.949453 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:34.950966 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:34.951570 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:34.951781 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:34.955418 (+ 3637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:34.955426 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:34.955440 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:34.955474 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:34.958945 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:34.959676 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:34.959691 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:34.959693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:34.959847 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:34.960267 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:34.960272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:34.960275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:34.960796 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:34.961011 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:34.961621 (+ 610us) server_negotiation.cc:299] Negotiation successful
0811 01:50:34.962050 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":670,"thread_start_us":448,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.125@KRBTEST.COM" created.
I20260811 01:50:34.967316 16442 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:50:34.968585 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:34.968129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:38023 (local address 127.0.0.1:34242)
0811 01:50:34.968356 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:34.968465 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:38023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 01:50:34.968874 16385 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.214.125:38023: connect: Connection refused (error 111)
I20260811 01:50:34.970587 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:34.970218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:38023 (local address 127.0.0.1:34256)
0811 01:50:34.970375 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:34.970448 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:38023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 01:50:34.970846 16442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.125:38023: Network error: Client connection negotiation failed: client connection to 127.21.214.125:38023: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.125.keytab.
Entry for principal kudu/127.21.214.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.125.keytab.
I20260811 01:50:35.008625 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.125:38023
--webserver_interface=127.21.214.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.125.keytab
--principal=kudu/127.21.214.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:39019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:44133,127.21.214.125:38023,127.21.214.124:41081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:50:35.020289 16442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } attempt: 1
I20260811 01:50:35.021642 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.021110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:38023 (local address 127.0.0.1:34262)
0811 01:50:35.021374 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.021503 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:38023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 01:50:35.022866 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.022358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:38023 (local address 127.0.0.1:34274)
0811 01:50:35.022544 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.022640 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:38023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 01:50:35.023113 16442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.125:38023: Network error: Client connection negotiation failed: client connection to 127.21.214.125:38023: connect: Connection refused (error 111)
I20260811 01:50:35.091446 16442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } attempt: 2
I20260811 01:50:35.092983 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.092357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:38023 (local address 127.0.0.1:34276)
0811 01:50:35.092665 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.092805 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:38023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 01:50:35.094161 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.093726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:38023 (local address 127.0.0.1:34284)
0811 01:50:35.093907 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.093993 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:38023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 01:50:35.094408 16442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.125:38023: Network error: Client connection negotiation failed: client connection to 127.21.214.125:38023: connect: Connection refused (error 111)
I20260811 01:50:35.185731 16442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } attempt: 3
I20260811 01:50:35.187129 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.186541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:38023 (local address 127.0.0.1:34300)
0811 01:50:35.186788 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.186909 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:38023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 01:50:35.188582 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.188022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:38023 (local address 127.0.0.1:34314)
0811 01:50:35.188237 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.188355 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:38023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 01:50:35.188968 16442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.125:38023: Network error: Client connection negotiation failed: client connection to 127.21.214.125:38023: connect: Connection refused (error 111)
I20260811 01:50:35.332264 16442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } attempt: 4
I20260811 01:50:35.333630 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.333067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:38023 (local address 127.0.0.1:34322)
0811 01:50:35.333390 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.333485 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:38023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193}
I20260811 01:50:35.334816 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.334412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:38023 (local address 127.0.0.1:34330)
0811 01:50:35.334605 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.334689 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.125:38023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260811 01:50:35.335003 16442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.125:38023: Network error: Client connection negotiation failed: client connection to 127.21.214.125:38023: connect: Connection refused (error 111)
W20260811 01:50:35.380499 16451 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 01:50:35.380935 16451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:35.381071 16451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:35.381256 16451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:35.389039 16451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:35.389223 16451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:35.389359 16451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:35.389462 16451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:35.389556 16451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:35.399962 16451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:39019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:44133,127.21.214.125:38023,127.21.214.124:41081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.125.keytab
--principal=kudu/127.21.214.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.125:38023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.214.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:35.401544 16451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:35.402968 16451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:35.415367 16457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:35.415324 16458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:35.415576 16460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:35.416455 16451 server_base.cc:1034] running on GCE node
I20260811 01:50:35.417337 16451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:35.418798 16451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:35.420039 16451 hybrid_clock.cc:648] HybridClock initialized: now 1786413035419965 us; error 45 us; skew 500 ppm
I20260811 01:50:35.420348 16451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:35 dist-test-slave-9gcw krb5kdc[16348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413035, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:35.423902 16451 init.cc:377] Logged in from keytab as kudu/127.21.214.125@KRBTEST.COM (short username kudu)
I20260811 01:50:35.425545 16451 webserver.cc:460] Webserver started at http://127.21.214.125:43139/ using document root <none> and password file <none>
I20260811 01:50:35.426326 16451 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:35.426451 16451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:35.426817 16451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:35.430092 16451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-1/data/instance:
uuid: "fa689e5b8ce94ecc979a6d37643effe5"
format_stamp: "Formatted at 2026-08-11 01:50:35 on dist-test-slave-9gcw"
server_key: "231d43c48af1b90c0333943b93eeb211"
server_key_iv: "ca2905012651faab149452f38cb6aee8"
server_key_version: "encryptionkey@0"
I20260811 01:50:35.430954 16451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-1/wal/instance:
uuid: "fa689e5b8ce94ecc979a6d37643effe5"
format_stamp: "Formatted at 2026-08-11 01:50:35 on dist-test-slave-9gcw"
server_key: "231d43c48af1b90c0333943b93eeb211"
server_key_iv: "ca2905012651faab149452f38cb6aee8"
server_key_version: "encryptionkey@0"
I20260811 01:50:35.436583 16451 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 01:50:35.440372 16467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:35.441872 16451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 01:50:35.442068 16451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-1/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-1/wal
uuid: "fa689e5b8ce94ecc979a6d37643effe5"
format_stamp: "Formatted at 2026-08-11 01:50:35 on dist-test-slave-9gcw"
server_key: "231d43c48af1b90c0333943b93eeb211"
server_key_iv: "ca2905012651faab149452f38cb6aee8"
server_key_version: "encryptionkey@0"
I20260811 01:50:35.442305 16451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:35.470588 16451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:35.475559 16451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:35.475907 16451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:35.495208 16451 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.125:38023
I20260811 01:50:35.495242 16528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.125:38023 every 8 connection(s)
I20260811 01:50:35.496726 16451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-1/data/info.pb
I20260811 01:50:35.500761 16529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:50:35.505281 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 16451
I20260811 01:50:35.505488 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-1/wal/instance
I20260811 01:50:35.506084 22361 external_mini_cluster.cc:1428] Setting key 093769eea0db93262919be11b9c4983b
I20260811 01:50:35.509286 16529 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:50:35 dist-test-slave-9gcw krb5kdc[16348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413035, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.125@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:50:35.541786 16531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.512921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60040 (local address 127.21.214.126:44133)
0811 01:50:35.522757 (+ 9836us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:35.522762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:35.522784 (+ 22us) server_negotiation.cc:407] Connection header received
0811 01:50:35.524553 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:35.524562 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:35.524664 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:35.524806 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:35.526568 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:35.527770 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:35.529383 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:35.530187 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:35.530341 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:35.534035 (+ 3694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:35.534044 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:35.534052 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:35.534115 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:35.538185 (+ 4070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:35.538977 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:35.538986 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:35.538993 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:35.539162 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:35.539662 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:35.539678 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:35.539683 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:35.540114 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:35.540295 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:35.541027 (+ 732us) server_negotiation.cc:299] Negotiation successful
0811 01:50:35.541319 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9711,"thread_start_us":9597,"threads_started":1}
WARNING: no policy specified for kudu/127.21.214.124@KRBTEST.COM; defaulting to no policy
I20260811 01:50:35.542529 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.513727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:44133 (local address 127.0.0.1:60040)
0811 01:50:35.521745 (+ 8018us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.521813 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 01:50:35.524191 (+ 2378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:50:35.525039 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:50:35.525048 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:50:35.525697 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:50:35.526297 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:35.526341 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:35.527940 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:35.527944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:35.529222 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:35.529246 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:35.530422 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:35.530427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:35.530653 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:35.531396 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:50:35.531419 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:50:35.533754 (+ 2335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:50:35.538339 (+ 4585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:35.538346 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:35.538360 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:35.538822 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:35.539283 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:35.539286 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:35.539288 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:35.539523 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:35.540246 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:50:35.540252 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:50:35.540620 (+ 368us) client_negotiation.cc:769] Sending connection context
0811 01:50:35.540857 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 01:50:35.541332 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7605,"thread_start_us":7296,"threads_started":1}
I20260811 01:50:35.544480 16529 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.214.124@KRBTEST.COM" created.
Aug 11 01:50:35 dist-test-slave-9gcw krb5kdc[16348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413035, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.125@KRBTEST.COM for kudu/127.21.214.125@KRBTEST.COM
I20260811 01:50:35.563575 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.545553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:38023 (local address 127.0.0.1:34332)
0811 01:50:35.545790 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.545812 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 01:50:35.545922 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:50:35.546828 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:50:35.546836 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:50:35.547420 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:50:35.547637 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:35.547651 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:35.549075 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:35.549083 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:35.550605 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:35.550624 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:35.551880 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:35.551888 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:35.552113 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:35.552875 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:50:35.552905 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:50:35.555940 (+ 3035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:50:35.560665 (+ 4725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:35.560679 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:35.560685 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:35.561266 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:35.561692 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:35.561697 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:35.561701 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:35.561883 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:35.562529 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:50:35.562538 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:50:35.562699 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 01:50:35.562850 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 01:50:35.563166 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 01:50:35.563694 16534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.545543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34332 (local address 127.21.214.125:38023)
0811 01:50:35.546401 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:35.546407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:35.546433 (+ 26us) server_negotiation.cc:407] Connection header received
0811 01:50:35.546477 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:35.546482 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:35.546656 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:35.546822 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:35.547824 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:35.548919 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:35.550762 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:35.551632 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:35.551847 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:35.556212 (+ 4365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:35.556235 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:35.556260 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:35.556317 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:35.560497 (+ 4180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:35.561429 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:35.561436 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:35.561438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:35.561569 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:35.562008 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:35.562018 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:35.562020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:35.562397 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:35.562642 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:35.563020 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 01:50:35.563300 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":700,"thread_start_us":510,"threads_started":1}
I20260811 01:50:35.566309 16529 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:50:35.567637 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.567135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53660)
0811 01:50:35.567369 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.567481 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260811 01:50:35.567982 16470 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
I20260811 01:50:35.569748 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.569366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53662)
0811 01:50:35.569552 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.569627 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 01:50:35.569998 16529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:41081: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.214.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.124.keytab.
Entry for principal kudu/127.21.214.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.124.keytab.
I20260811 01:50:35.589365 22361 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw71eKA/build/asan/bin/kudu
/tmp/dist-test-taskw71eKA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.124:41081
--webserver_interface=127.21.214.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.124.keytab
--principal=kudu/127.21.214.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.214.84:39019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.214.126:44133,127.21.214.125:38023,127.21.214.124:41081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 01:50:35.619330 16529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } attempt: 1
I20260811 01:50:35.620934 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.620243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53668)
0811 01:50:35.620527 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.620685 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 01:50:35.622418 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.621900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53674)
0811 01:50:35.622122 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.622242 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 01:50:35.622833 16529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:41081: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
I20260811 01:50:35.634310 16442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } attempt: 5
Aug 11 01:50:35 dist-test-slave-9gcw krb5kdc[16348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413034, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for kudu/127.21.214.125@KRBTEST.COM
I20260811 01:50:35.649951 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.635196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:38023 (local address 127.0.0.1:34334)
0811 01:50:35.635470 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.635497 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 01:50:35.635663 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:50:35.636064 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:50:35.636067 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:50:35.636510 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:50:35.636812 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:35.636826 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:35.638269 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:35.638274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:35.639190 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:35.639202 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:35.640211 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:35.640216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:35.640389 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:35.641100 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:50:35.641119 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:50:35.643344 (+ 2225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:50:35.647353 (+ 4009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:35.647357 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:35.647360 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:35.647779 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:35.648229 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:35.648232 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:35.648234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:35.648382 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:35.649060 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:50:35.649063 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:50:35.649280 (+ 217us) client_negotiation.cc:769] Sending connection context
0811 01:50:35.649403 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 01:50:35.649662 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 01:50:35.650182 16534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.635464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34334 (local address 127.21.214.125:38023)
0811 01:50:35.635703 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:35.635710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:35.635734 (+ 24us) server_negotiation.cc:407] Connection header received
0811 01:50:35.635813 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:35.635818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:35.635898 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:35.636021 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:35.636994 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:35.638088 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:35.639361 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:35.640057 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:35.640196 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:35.643572 (+ 3376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:35.643579 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:35.643584 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:35.643627 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:35.647134 (+ 3507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:35.647918 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:35.647926 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:35.647931 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:35.648109 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:35.648498 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:35.648503 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:35.648506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:35.648935 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:35.649100 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:35.649569 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 01:50:35.649830 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 01:50:35.651304 16442 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } has no permanent_uuid. Determining permanent_uuid...
I20260811 01:50:35.652384 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.651986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53684)
0811 01:50:35.652160 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.652251 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 01:50:35.653467 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.653007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53692)
0811 01:50:35.653220 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.653316 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 01:50:35.653692 16442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:41081: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
I20260811 01:50:35.691107 16529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } attempt: 2
I20260811 01:50:35.692510 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.691931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53702)
0811 01:50:35.692186 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.692308 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 01:50:35.693787 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.693369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53716)
0811 01:50:35.693538 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.693634 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 01:50:35.694093 16529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:41081: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
I20260811 01:50:35.704949 16442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } attempt: 1
I20260811 01:50:35.706363 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.705729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53718)
0811 01:50:35.706033 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.706153 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260811 01:50:35.707603 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.707194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53724)
0811 01:50:35.707354 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.707444 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 01:50:35.707870 16442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:41081: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
I20260811 01:50:35.776125 16442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } attempt: 2
I20260811 01:50:35.777702 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.776941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53734)
0811 01:50:35.777286 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.777441 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 01:50:35.779122 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.778574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53742)
0811 01:50:35.778779 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.778905 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 01:50:35.779444 16442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:41081: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
I20260811 01:50:35.785323 16529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } attempt: 3
I20260811 01:50:35.786470 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.785966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53754)
0811 01:50:35.786173 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.786280 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 01:50:35.787587 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.787109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53768)
0811 01:50:35.787348 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.787442 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
W20260811 01:50:35.787840 16529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:41081: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
I20260811 01:50:35.885725 16442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } attempt: 3
I20260811 01:50:35.887061 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.886429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53770)
0811 01:50:35.886693 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.886826 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 01:50:35.888248 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.887802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53772)
0811 01:50:35.888012 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.888107 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260811 01:50:35.888501 16442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:41081: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
I20260811 01:50:35.931098 16529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } attempt: 4
I20260811 01:50:35.932412 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.931796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53788)
0811 01:50:35.932066 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.932179 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 01:50:35.933516 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:35.933100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53800)
0811 01:50:35.933283 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:35.933376 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 01:50:35.933740 16529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.214.124:41081: Network error: Client connection negotiation failed: client connection to 127.21.214.124:41081: connect: Connection refused (error 111)
W20260811 01:50:35.960987 16537 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 01:50:35.961452 16537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 01:50:35.961567 16537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 01:50:35.961702 16537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 01:50:35.969189 16537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 01:50:35.969298 16537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 01:50:35.969422 16537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 01:50:35.969511 16537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 01:50:35.969611 16537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 01:50:35.980188 16537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.214.84:39019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.214.126:44133,127.21.214.125:38023,127.21.214.124:41081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/krb5kdc/kudu_127.21.214.124.keytab
--principal=kudu/127.21.214.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.214.124:41081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.214.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 00:28:44 UTC on ccycloud.quasar-xgjmlp.root.comops.site
build id 4579415
ASAN enabled
I20260811 01:50:35.981664 16537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 01:50:35.983120 16537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 01:50:35.996047 16543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:35.996078 16545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 01:50:35.996112 16542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 01:50:35.998927 16537 server_base.cc:1034] running on GCE node
I20260811 01:50:35.999794 16537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 01:50:36.001379 16537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 01:50:36.002643 16537 hybrid_clock.cc:648] HybridClock initialized: now 1786413036002572 us; error 46 us; skew 500 ppm
I20260811 01:50:36.003019 16537 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 01:50:36 dist-test-slave-9gcw krb5kdc[16348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413036, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 01:50:36.006532 16537 init.cc:377] Logged in from keytab as kudu/127.21.214.124@KRBTEST.COM (short username kudu)
I20260811 01:50:36.008286 16537 webserver.cc:460] Webserver started at http://127.21.214.124:37761/ using document root <none> and password file <none>
I20260811 01:50:36.009088 16537 fs_manager.cc:359] Metadata directory not provided
I20260811 01:50:36.009253 16537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 01:50:36.009629 16537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 01:50:36.012799 16537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-2/data/instance:
uuid: "8eca5063a71b4e929812442425c8ced8"
format_stamp: "Formatted at 2026-08-11 01:50:36 on dist-test-slave-9gcw"
server_key: "ebf01a16a23ae384ddb06ba79ef249bb"
server_key_iv: "8dc391831187417f8e223c5f04e38126"
server_key_version: "encryptionkey@0"
I20260811 01:50:36.013700 16537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-2/wal/instance:
uuid: "8eca5063a71b4e929812442425c8ced8"
format_stamp: "Formatted at 2026-08-11 01:50:36 on dist-test-slave-9gcw"
server_key: "ebf01a16a23ae384ddb06ba79ef249bb"
server_key_iv: "8dc391831187417f8e223c5f04e38126"
server_key_version: "encryptionkey@0"
I20260811 01:50:36.019230 16537 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 01:50:36.022969 16553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 01:50:36.024358 16537 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 01:50:36.024562 16537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-2/data,/tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-2/wal
uuid: "8eca5063a71b4e929812442425c8ced8"
format_stamp: "Formatted at 2026-08-11 01:50:36 on dist-test-slave-9gcw"
server_key: "ebf01a16a23ae384ddb06ba79ef249bb"
server_key_iv: "8dc391831187417f8e223c5f04e38126"
server_key_version: "encryptionkey@0"
I20260811 01:50:36.024801 16537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 01:50:36.036648 16537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 01:50:36.041385 16537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 01:50:36.041682 16537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 01:50:36.061476 16537 rpc_server.cc:307] RPC server started. Bound to: 127.21.214.124:41081
I20260811 01:50:36.061498 16609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.214.124:41081 every 8 connection(s)
I20260811 01:50:36.062965 16537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-2/data/info.pb
I20260811 01:50:36.065297 22361 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw71eKA/build/asan/bin/kudu as pid 16537
I20260811 01:50:36.065531 22361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw71eKA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786412636131983-22361-0/minicluster-data/master-2/wal/instance
I20260811 01:50:36.065730 16442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } attempt: 4
I20260811 01:50:36.066108 22361 external_mini_cluster.cc:1428] Setting key c1da303c8810c9aef79a418db4d86391
I20260811 01:50:36.067667 16610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 01:50:36.078027 16610 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:50:36 dist-test-slave-9gcw krb5kdc[16348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.124@KRBTEST.COM
Aug 11 01:50:36 dist-test-slave-9gcw krb5kdc[16348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413034, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.126@KRBTEST.COM for kudu/127.21.214.124@KRBTEST.COM
I20260811 01:50:36.093577 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:36.066473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53804)
0811 01:50:36.066767 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:36.066791 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 01:50:36.066936 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:50:36.071277 (+ 4341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:50:36.071282 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:50:36.071868 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:50:36.072118 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:36.072131 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.074572 (+ 2441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.074579 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:36.076061 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:36.076078 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.078115 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.078121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:36.078347 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:36.081022 (+ 2675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:50:36.081054 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:50:36.084647 (+ 3593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:50:36.090222 (+ 5575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:36.090229 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:36.090235 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:36.090823 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:36.091392 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:36.091398 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:36.091401 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:36.091568 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:36.092473 (+ 905us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:50:36.092478 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:50:36.092662 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 01:50:36.092825 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 01:50:36.093147 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":1}
I20260811 01:50:36.095273 16613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:36.067594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53804 (local address 127.21.214.124:41081)
0811 01:50:36.068526 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.068546 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.068603 (+ 57us) server_negotiation.cc:407] Connection header received
0811 01:50:36.070410 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.070430 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.070916 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.071319 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.072302 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.074402 (+ 2100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.076220 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.077287 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.077538 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:36.085023 (+ 7485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:36.085033 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:36.085040 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:36.085095 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:36.090058 (+ 4963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.090978 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.090983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.090987 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.091246 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.091707 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.091712 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.091715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.092308 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:36.092518 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:36.093099 (+ 581us) server_negotiation.cc:299] Negotiation successful
0811 01:50:36.093648 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":146,"server-negotiator.queue_time_us":680,"spinlock_wait_cycles":34944,"thread_start_us":428,"threads_started":1}
I20260811 01:50:36.095229 16614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:36.068461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53818 (local address 127.21.214.124:41081)
0811 01:50:36.069079 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.069084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.069115 (+ 31us) server_negotiation.cc:407] Connection header received
0811 01:50:36.070431 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.070439 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.070921 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.072000 (+ 1079us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.073266 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.074674 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.075958 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.077287 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.077538 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:36.083873 (+ 6335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:36.083887 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:36.083915 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:36.083982 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:36.090156 (+ 6174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.090953 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.090961 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.090978 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.091234 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.091671 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.091677 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.091680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.092303 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:36.094163 (+ 1860us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:36.094324 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 01:50:36.094548 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":274,"server-negotiator.queue_time_us":484,"spinlock_wait_cycles":20352,"thread_start_us":284,"threads_started":1}
Aug 11 01:50:36 dist-test-slave-9gcw krb5kdc[16348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413036, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.124@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:50:36.103382 16442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425: Bootstrap starting.
I20260811 01:50:36.106495 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:36.081429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.126:44133 (local address 127.0.0.1:60048)
0811 01:50:36.087008 (+ 5579us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:36.087070 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 01:50:36.087283 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:50:36.087895 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:50:36.087902 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:50:36.088627 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:50:36.088941 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:36.088950 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.090193 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.090199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:36.092100 (+ 1901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:36.092114 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.093486 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.093494 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:36.093731 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:36.094457 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:50:36.094474 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:50:36.098263 (+ 3789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:50:36.103450 (+ 5187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:36.103466 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:36.103475 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:36.104068 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:36.104482 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:36.104490 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:36.104493 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:36.104648 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:36.105326 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:50:36.105334 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:50:36.105509 (+ 175us) client_negotiation.cc:769] Sending connection context
0811 01:50:36.105708 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 01:50:36.106070 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5257,"thread_start_us":5055,"threads_started":1}
I20260811 01:50:36.107489 16442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:36.107954 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:36.081491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60048 (local address 127.21.214.126:44133)
0811 01:50:36.087569 (+ 6078us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.087574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.087588 (+ 14us) server_negotiation.cc:407] Connection header received
0811 01:50:36.087631 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.087635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.087692 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.087827 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.089108 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.089958 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.092275 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.093029 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.093188 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:36.098517 (+ 5329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:36.098522 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:36.098525 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:36.098591 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:36.103235 (+ 4644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.104175 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.104181 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.104185 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.104351 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.104770 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.104777 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.104780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.105117 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:36.105370 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:36.107379 (+ 2009us) server_negotiation.cc:299] Negotiation successful
0811 01:50:36.107603 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":5950,"thread_start_us":4701,"threads_started":1}
I20260811 01:50:36.109079 16442 log.cc:824] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425: Log is configured to *not* fsync() on all Append() calls
I20260811 01:50:36.109741 16610 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:50:36 dist-test-slave-9gcw krb5kdc[16348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.126@KRBTEST.COM
I20260811 01:50:36.112589 16442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425: No bootstrap required, opened a new log
Aug 11 01:50:36 dist-test-slave-9gcw krb5kdc[16348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413036, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.124@KRBTEST.COM for kudu/127.21.214.125@KRBTEST.COM
I20260811 01:50:36.121484 16618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:36.098770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60064 (local address 127.21.214.126:44133)
0811 01:50:36.102066 (+ 3296us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.102072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.102101 (+ 29us) server_negotiation.cc:407] Connection header received
0811 01:50:36.102158 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.102162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.102238 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.102370 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.104170 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.105589 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.107279 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.108104 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.108869 (+ 765us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:36.113724 (+ 4855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:36.113730 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:36.113733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:36.113770 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:36.117985 (+ 4215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.118911 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.118919 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.118924 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.119124 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.119670 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.119678 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.119683 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.120089 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:36.120419 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:36.120724 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 01:50:36.120966 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3145,"thread_start_us":2910,"threads_started":1}
I20260811 01:50:36.123463 16442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } } peers { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } } peers { permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } }
I20260811 01:50:36.124665 16442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e3f9cf0caa94cee9abeef78d2b61425, State: Initialized, Role: FOLLOWER
I20260811 01:50:36.125298 16442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } } peers { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } } peers { permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } }
I20260811 01:50:36.126559 16621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } } peers { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } } peers { permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } } }
I20260811 01:50:36.127204 16621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:50:36.127745 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:36.110668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.125:38023 (local address 127.0.0.1:34338)
0811 01:50:36.111097 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:36.111120 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:50:36.111253 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:50:36.111647 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:50:36.111654 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:50:36.112175 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:50:36.112384 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:36.112397 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.113876 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.113883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:36.115264 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:36.115280 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.116351 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.116358 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:36.116583 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:36.117223 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:50:36.117247 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:50:36.119873 (+ 2626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:50:36.124290 (+ 4417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:36.124296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:36.124300 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:36.124938 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:36.125430 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:36.125438 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:36.125443 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:36.125633 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:36.126589 (+ 956us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:50:36.126594 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:50:36.126792 (+ 198us) client_negotiation.cc:769] Sending connection context
0811 01:50:36.126962 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 01:50:36.127331 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280}
I20260811 01:50:36.128261 16442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:50:36.129073 16534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:36.110679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34338 (local address 127.21.214.125:38023)
0811 01:50:36.110915 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.110923 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.111229 (+ 306us) server_negotiation.cc:407] Connection header received
0811 01:50:36.111387 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.111393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.111481 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.111593 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.112569 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.113686 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.115456 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.116150 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.116280 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:36.120086 (+ 3806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:36.120092 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:36.120097 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:36.120141 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:36.124123 (+ 3982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.125080 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.125088 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.125091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.125288 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.125878 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.125888 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.125895 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.126403 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:36.128200 (+ 1797us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:36.128351 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 01:50:36.128628 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 01:50:36.130467 16610 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 01:50:36 dist-test-slave-9gcw krb5kdc[16348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.214.125@KRBTEST.COM
Aug 11 01:50:36 dist-test-slave-9gcw krb5kdc[16348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413036, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.124@KRBTEST.COM for kudu/127.21.214.124@KRBTEST.COM
I20260811 01:50:36.146620 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:36.123809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34354 (local address 127.21.214.125:38023)
0811 01:50:36.124476 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.124499 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.124523 (+ 24us) server_negotiation.cc:407] Connection header received
0811 01:50:36.124572 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.124576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.124652 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.126919 (+ 2267us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.127235 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.128484 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.130572 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.131299 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.131437 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:36.137762 (+ 6325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:36.137770 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:36.137775 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:36.137823 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:36.142153 (+ 4330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.143449 (+ 1296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.143460 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.143466 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.143643 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.144261 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.144267 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.144272 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.144740 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:36.145051 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:36.145815 (+ 764us) server_negotiation.cc:299] Negotiation successful
0811 01:50:36.146109 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":293,"threads_started":1}
I20260811 01:50:36.152755 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:36.131474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53828)
0811 01:50:36.131774 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:36.131797 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 01:50:36.131957 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:50:36.132766 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:50:36.132774 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:50:36.133508 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:50:36.133732 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:36.133750 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.135383 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.135392 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:36.137600 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:36.137623 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.138830 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.138858 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:36.139107 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:36.139763 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:50:36.139794 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:50:36.142674 (+ 2880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:50:36.149445 (+ 6771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:36.149450 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:36.149455 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:36.150040 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:36.150551 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:36.150557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:36.150561 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:36.150738 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:36.151589 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:50:36.151596 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:50:36.151763 (+ 167us) client_negotiation.cc:769] Sending connection context
0811 01:50:36.151954 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 01:50:36.152297 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
W20260811 01:50:36.153365 16390 tablet.cc:2369] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:50:36.154313 16614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:36.131559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53828 (local address 127.21.214.124:41081)
0811 01:50:36.131776 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.131783 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.131896 (+ 113us) server_negotiation.cc:407] Connection header received
0811 01:50:36.132107 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.132115 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.132208 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.132708 (+ 500us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.133927 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.135164 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.137771 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.138503 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.138660 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:36.142861 (+ 4201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:36.142868 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:36.142873 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:36.142916 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:36.149260 (+ 6344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.150210 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.150216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.150219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.150405 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.150873 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.150878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.150882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.151339 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:36.151529 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:36.153303 (+ 1774us) server_negotiation.cc:299] Negotiation successful
0811 01:50:36.153912 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 01:50:36.155364 22361 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 01:50:36.160929 16610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8: Bootstrap starting.
I20260811 01:50:36.161536 16413 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6e3f9cf0caa94cee9abeef78d2b61425"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60064
I20260811 01:50:36.162019 16413 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 01:50:36.162266 16413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [term 0 FOLLOWER]: Advancing to term 1
W20260811 01:50:36.164213 16633 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 01:50:36.164347 16633 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 01:50:36.153551 16365 master_runner.cc:186] Error getting master registration for 127.21.214.125:38023: OK, instance_id { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" instance_seqno: 1786413035489617 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 01:50:36.166754 16365 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:50:36.166525 16413 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } } peers { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } } peers { permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } }
I20260811 01:50:36.168241 16413 leader_election.cc:288] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [CANDIDATE]: Term 1 election: Requested vote from peers fa689e5b8ce94ecc979a6d37643effe5 (127.21.214.125:38023), 8eca5063a71b4e929812442425c8ced8 (127.21.214.124:41081)
I20260811 01:50:36.169113 16610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:36.170755 16610 log.cc:824] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8: Log is configured to *not* fsync() on all Append() calls
I20260811 01:50:36.175602 16610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8: No bootstrap required, opened a new log
I20260811 01:50:36.188357 16610 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } } peers { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } } peers { permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } }
I20260811 01:50:36.189980 16610 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eca5063a71b4e929812442425c8ced8, State: Initialized, Role: FOLLOWER
I20260811 01:50:36.190670 16610 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } } peers { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } } peers { permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } }
I20260811 01:50:36.218451 16610 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:50:36.218595 16583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8eca5063a71b4e929812442425c8ced8"
I20260811 01:50:36.219249 16583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:50:36.225293 16583 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e3f9cf0caa94cee9abeef78d2b61425 in term 1.
I20260811 01:50:36.226250 16387 leader_election.cc:302] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e3f9cf0caa94cee9abeef78d2b61425, 8eca5063a71b4e929812442425c8ced8; no voters:
I20260811 01:50:36.226985 16621 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 01:50:36.227011 16646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } } peers { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } } peers { permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } } }
I20260811 01:50:36.232968 16529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } attempt: 5
I20260811 01:50:36.227528 16646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8 [sys.catalog]: This master's current role is: FOLLOWER
Aug 11 01:50:36 dist-test-slave-9gcw krb5kdc[16348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786413035, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.214.125@KRBTEST.COM for kudu/127.21.214.124@KRBTEST.COM
I20260811 01:50:36.252385 16621 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [term 1 LEADER]: Becoming Leader. State: Replica: 6e3f9cf0caa94cee9abeef78d2b61425, State: Running, Role: LEADER
I20260811 01:50:36.253855 16621 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } } peers { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } } peers { permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } }
I20260811 01:50:36.270130 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:36.186890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34362 (local address 127.21.214.125:38023)
0811 01:50:36.187118 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.187124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.195144 (+ 8020us) server_negotiation.cc:407] Connection header received
0811 01:50:36.197220 (+ 2076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.197230 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.197340 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.197457 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.223681 (+ 26224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.224912 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.237808 (+ 12896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.238552 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.238688 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:36.254235 (+ 15547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:36.254247 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:36.254255 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:36.254304 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:36.259522 (+ 5218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.265654 (+ 6132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.265663 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.265669 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.265904 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.266933 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.266942 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.266947 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.267399 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:36.267761 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:36.268456 (+ 695us) server_negotiation.cc:299] Negotiation successful
0811 01:50:36.269575 (+ 1119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":808,"server-negotiator.queue_time_us":75}
I20260811 01:50:36.272358 16618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:36.186367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60072 (local address 127.21.214.126:44133)
0811 01:50:36.186605 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.186615 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.204982 (+ 18367us) server_negotiation.cc:407] Connection header received
0811 01:50:36.206815 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.206826 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.206963 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.207066 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.235046 (+ 27980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.236582 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.245011 (+ 8429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.246163 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.246316 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:36.256624 (+ 10308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:36.256631 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:36.256636 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:36.256704 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:36.260990 (+ 4286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.267022 (+ 6032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.267033 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.267039 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.267270 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.268933 (+ 1663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.268941 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.268945 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.269508 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:36.269852 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:36.271423 (+ 1571us) server_negotiation.cc:299] Negotiation successful
0811 01:50:36.271794 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 01:50:36.272464 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:36.233788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.214.124:41081 (local address 127.0.0.1:53840)
0811 01:50:36.234293 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 01:50:36.234317 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 01:50:36.234459 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 01:50:36.235679 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 01:50:36.235687 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 01:50:36.236683 (+ 996us) client_negotiation.cc:189] Negotiated authn=SASL
0811 01:50:36.236995 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:36.237014 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.238987 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.238996 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:36.241482 (+ 2486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 01:50:36.241505 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.243135 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.243144 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 01:50:36.243529 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:36.244417 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 01:50:36.244452 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 01:50:36.248885 (+ 4433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 01:50:36.254553 (+ 5668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:36.254567 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:36.254574 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:36.260204 (+ 5630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:36.261242 (+ 1038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 01:50:36.261249 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 01:50:36.261252 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 01:50:36.261565 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 01:50:36.266084 (+ 4519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 01:50:36.266093 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 01:50:36.266347 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 01:50:36.266533 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 01:50:36.271948 (+ 5415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329}
I20260811 01:50:36.272934 16613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:36.234735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53840 (local address 127.21.214.124:41081)
0811 01:50:36.234979 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.234989 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.235019 (+ 30us) server_negotiation.cc:407] Connection header received
0811 01:50:36.235103 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.235111 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.235216 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.235396 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.237253 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.238764 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.241728 (+ 2964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.242641 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.242816 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:36.249214 (+ 6398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:36.249226 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:36.249234 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:36.249294 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:36.254293 (+ 4999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.260403 (+ 6110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.260412 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.260418 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.260647 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.261770 (+ 1123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.261779 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.261786 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.262320 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:36.262532 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:36.268063 (+ 5531us) server_negotiation.cc:299] Negotiation successful
0811 01:50:36.268797 (+ 734us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 01:50:36.280037 16529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5: Bootstrap starting.
I20260811 01:50:36.283598 16614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:36.191236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53830 (local address 127.21.214.124:41081)
0811 01:50:36.191432 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.191440 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.221726 (+ 30286us) server_negotiation.cc:407] Connection header received
0811 01:50:36.223144 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.223151 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.223298 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.223387 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.240661 (+ 17274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.242222 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.254170 (+ 11948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:36.255117 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:36.255232 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:36.274259 (+ 19027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 01:50:36.274265 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 01:50:36.274270 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 01:50:36.274318 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 01:50:36.278810 (+ 4492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.279810 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.279818 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.279822 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.280009 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 01:50:36.280591 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 01:50:36.280596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 01:50:36.280599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 01:50:36.281107 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 01:50:36.281419 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:36.282854 (+ 1435us) server_negotiation.cc:299] Negotiation successful
0811 01:50:36.283143 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 01:50:36.287725 16649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e3f9cf0caa94cee9abeef78d2b61425. Latest consensus state: current_term: 1 leader_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } } peers { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } } peers { permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } } }
I20260811 01:50:36.288216 16649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:36.289296 16656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260811 01:50:36.293897 16561 tablet.cc:2369] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 01:50:36.294291 16529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5: Neither blocks nor log segments found. Creating new log.
I20260811 01:50:36.295919 16529 log.cc:824] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5: Log is configured to *not* fsync() on all Append() calls
I20260811 01:50:36.300437 16529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5: No bootstrap required, opened a new log
I20260811 01:50:36.305646 16656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 01:50:36.314313 16529 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } } peers { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } } peers { permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } }
I20260811 01:50:36.314801 16529 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa689e5b8ce94ecc979a6d37643effe5, State: Initialized, Role: FOLLOWER
I20260811 01:50:36.315451 16529 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } } peers { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } } peers { permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } }
W20260811 01:50:36.328019 16665 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 01:50:36.328303 16665 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 01:50:36.313303 16537 master_runner.cc:186] Error getting master registration for 127.21.214.125:38023: OK, instance_id { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" instance_seqno: 1786413035489617 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 01:50:36.332108 16537 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 01:50:36.336234 16668 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } } peers { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } } peers { permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } } }
I20260811 01:50:36.337037 16668 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:50:36.337946 16583 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8 [term 1 FOLLOWER]: Refusing update from remote peer 6e3f9cf0caa94cee9abeef78d2b61425: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:50:36.339469 16649 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:50:36.344874 16498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 01:50:36.345255 16499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fa689e5b8ce94ecc979a6d37643effe5"
I20260811 01:50:36.349749 16498 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5 [term 1 FOLLOWER]: Refusing update from remote peer 6e3f9cf0caa94cee9abeef78d2b61425: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 01:50:36.353571 16499 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e3f9cf0caa94cee9abeef78d2b61425 in term 1.
I20260811 01:50:36.355408 16529 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 01:50:36.356230 16649 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 01:50:36.369107 16646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e3f9cf0caa94cee9abeef78d2b61425. Latest consensus state: current_term: 1 leader_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } } peers { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } } peers { permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } } }
I20260811 01:50:36.371295 16646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:50:36.378823 16667 mvcc.cc:205] Tried to move back new op lower bound from 7317147796806541312 to 7317147796514881536. Current Snapshot: MvccSnapshot[applied={T|T < 7317147796806541312}]
I20260811 01:50:36.381245 16646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } } peers { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } } peers { permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } } }
I20260811 01:50:36.381572 16646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:50:36.403010 16621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } } peers { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } } peers { permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } } }
I20260811 01:50:36.403437 16621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:36.409821 16668 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e3f9cf0caa94cee9abeef78d2b61425. Latest consensus state: current_term: 1 leader_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } } peers { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } } peers { permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } } }
I20260811 01:50:36.410229 16668 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:50:36.416554 16677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } } peers { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } } peers { permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } } }
I20260811 01:50:36.417054 16677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425 [sys.catalog]: This master's current role is: LEADER
I20260811 01:50:36.425189 16670 mvcc.cc:205] Tried to move back new op lower bound from 7317147796806541312 to 7317147796514881536. Current Snapshot: MvccSnapshot[applied={T|T < 7317147796806541312 or (T in {7317147796806541312})}]
I20260811 01:50:36.429311 16672 mvcc.cc:205] Tried to move back new op lower bound from 7317147796806541312 to 7317147796514881536. Current Snapshot: MvccSnapshot[applied={T|T < 7317147796806541312}]
I20260811 01:50:36.432400 16656 catalog_manager.cc:1347] Generated new cluster ID: b32cc2c1514b4f54bc7d673257930013
I20260811 01:50:36.432554 16656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 01:50:36.435964 16668 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3f9cf0caa94cee9abeef78d2b61425" member_type: VOTER last_known_addr { host: "127.21.214.126" port: 44133 } } peers { permanent_uuid: "fa689e5b8ce94ecc979a6d37643effe5" member_type: VOTER last_known_addr { host: "127.21.214.125" port: 38023 } } peers { permanent_uuid: "8eca5063a71b4e929812442425c8ced8" member_type: VOTER last_known_addr { host: "127.21.214.124" port: 41081 } } }
I20260811 01:50:36.436273 16668 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 01:50:36.488087 16656 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 01:50:36.490490 16656 catalog_manager.cc:1504] Loading token signing keys...
I20260811 01:50:36.504101 16692 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 01:50:36.537771 16656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e3f9cf0caa94cee9abeef78d2b61425: Generated new TSK 0
I20260811 01:50:36.539127 16656 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 01:50:36.648856 16620 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:36.636046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34376 (local address 127.21.214.125:38023)
0811 01:50:36.636274 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.636279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.646036 (+ 9757us) server_negotiation.cc:407] Connection header received
0811 01:50:36.646287 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.646298 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.646420 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.646542 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.648504 (+ 1962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34376: BlockingRecv error: recv got EOF from 127.0.0.1:34376 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260811 01:50:36.649917 16618 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:36.635528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60078 (local address 127.21.214.126:44133)
0811 01:50:36.635815 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.635824 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.646990 (+ 11166us) server_negotiation.cc:407] Connection header received
0811 01:50:36.647210 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.647215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.647310 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.647428 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.649643 (+ 2215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60078: BlockingRecv error: recv got EOF from 127.0.0.1:60078 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 01:50:36.651103 16614 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:36.642522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53846 (local address 127.21.214.124:41081)
0811 01:50:36.642718 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.642724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.648185 (+ 5461us) server_negotiation.cc:407] Connection header received
0811 01:50:36.648342 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.648348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.648439 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.648544 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.650827 (+ 2283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53846: BlockingRecv error: recv got EOF from 127.0.0.1:53846 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 01:50:36.658126 16620 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:36.655215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34384 (local address 127.21.214.125:38023)
0811 01:50:36.655734 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.655741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.656195 (+ 454us) server_negotiation.cc:407] Connection header received
0811 01:50:36.656549 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.656603 (+ 54us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.656785 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.656945 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.657890 (+ 945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34384: BlockingRecv error: recv got EOF from 127.0.0.1:34384 (error 108)
Metrics: {"server-negotiator.queue_time_us":384}
W20260811 01:50:36.674767 16620 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:36.663320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34386 (local address 127.21.214.125:38023)
0811 01:50:36.663504 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.663510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.672896 (+ 9386us) server_negotiation.cc:407] Connection header received
0811 01:50:36.673109 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.673117 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.673277 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.673437 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.674492 (+ 1055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34386: BlockingRecv error: recv got EOF from 127.0.0.1:34386 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 01:50:36.687937 16620 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:36.685394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34388 (local address 127.21.214.125:38023)
0811 01:50:36.685624 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.685631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.686253 (+ 622us) server_negotiation.cc:407] Connection header received
0811 01:50:36.686477 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.686483 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.686590 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.686730 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.687681 (+ 951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34388: BlockingRecv error: recv got EOF from 127.0.0.1:34388 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 01:50:36.698920 16620 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:36.696226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34402 (local address 127.21.214.125:38023)
0811 01:50:36.696447 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.696455 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.697051 (+ 596us) server_negotiation.cc:407] Connection header received
0811 01:50:36.697377 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.697383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.697506 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.697644 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.698615 (+ 971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34402: BlockingRecv error: recv got EOF from 127.0.0.1:34402 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260811 01:50:36.717787 16620 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:36.715096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34406 (local address 127.21.214.125:38023)
0811 01:50:36.715351 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.715359 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.715984 (+ 625us) server_negotiation.cc:407] Connection header received
0811 01:50:36.716241 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.716247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.716387 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.716511 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.717557 (+ 1046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34406: BlockingRecv error: recv got EOF from 127.0.0.1:34406 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260811 01:50:36.741000 16620 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:36.738191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34422 (local address 127.21.214.125:38023)
0811 01:50:36.738424 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.738431 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.739112 (+ 681us) server_negotiation.cc:407] Connection header received
0811 01:50:36.739382 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.739389 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.739500 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.739646 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.740673 (+ 1027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34422: BlockingRecv error: recv got EOF from 127.0.0.1:34422 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 01:50:36.780308 16620 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:36.777452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34434 (local address 127.21.214.125:38023)
0811 01:50:36.777691 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.777699 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.778382 (+ 683us) server_negotiation.cc:407] Connection header received
0811 01:50:36.778642 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.778650 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.778764 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.778900 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.779984 (+ 1084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34434: BlockingRecv error: recv got EOF from 127.0.0.1:34434 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 01:50:36.851029 16620 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:36.848885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34450 (local address 127.21.214.125:38023)
0811 01:50:36.849112 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.849120 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.849507 (+ 387us) server_negotiation.cc:407] Connection header received
0811 01:50:36.849725 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.849731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.849836 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.849980 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.850774 (+ 794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34450: BlockingRecv error: recv got EOF from 127.0.0.1:34450 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 01:50:36.993026 16620 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:36.991060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34458 (local address 127.21.214.125:38023)
0811 01:50:36.991327 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:36.991335 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:36.991574 (+ 239us) server_negotiation.cc:407] Connection header received
0811 01:50:36.991785 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:36.991789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:36.991894 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:36.991985 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:36.992781 (+ 796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34458: BlockingRecv error: recv got EOF from 127.0.0.1:34458 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260811 01:50:37.255002 16620 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 01:50:37.252826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34460 (local address 127.21.214.125:38023)
0811 01:50:37.253079 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:37.253086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:37.253327 (+ 241us) server_negotiation.cc:407] Connection header received
0811 01:50:37.253577 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:37.253589 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:37.253712 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:37.253848 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 01:50:37.254720 (+ 872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34460: BlockingRecv error: recv got EOF from 127.0.0.1:34460 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 01:50:37.330672 16665 catalog_manager.cc:1259] Loaded cluster ID: b32cc2c1514b4f54bc7d673257930013
I20260811 01:50:37.330785 16665 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8: loading cluster ID for follower catalog manager: success
I20260811 01:50:37.333443 16665 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8: acquiring CA information for follower catalog manager: success
I20260811 01:50:37.335525 16665 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8eca5063a71b4e929812442425c8ced8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:50:37.508893 16692 catalog_manager.cc:1259] Loaded cluster ID: b32cc2c1514b4f54bc7d673257930013
I20260811 01:50:37.509099 16692 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5: loading cluster ID for follower catalog manager: success
I20260811 01:50:37.512162 16692 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5: acquiring CA information for follower catalog manager: success
I20260811 01:50:37.514317 16692 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fa689e5b8ce94ecc979a6d37643effe5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 01:50:37.781244 16766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:37.774296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34472 (local address 127.21.214.125:38023)
0811 01:50:37.775180 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:37.775204 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:37.775221 (+ 17us) server_negotiation.cc:407] Connection header received
0811 01:50:37.775288 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:37.775292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:37.775468 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:37.775648 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:50:37.776228 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:37.777227 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:37.778627 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:37.779152 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:37.779336 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:37.779897 (+ 561us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:50:37.780516 (+ 619us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:50:37.780695 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:37.780751 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 01:50:37.780853 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":529,"threads_started":1}
I20260811 01:50:37.807003 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:37.799403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53848 (local address 127.21.214.124:41081)
0811 01:50:37.799937 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:37.799944 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:37.800303 (+ 359us) server_negotiation.cc:407] Connection header received
0811 01:50:37.800603 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:37.800611 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:37.800827 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:37.801034 (+ 207us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:50:37.801526 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:37.802701 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:37.804123 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:37.804781 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:37.804933 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:37.805473 (+ 540us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:50:37.806286 (+ 813us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:50:37.806494 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:37.806555 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 01:50:37.806648 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":255,"threads_started":1}
I20260811 01:50:37.823485 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:37.815241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53854 (local address 127.21.214.124:41081)
0811 01:50:37.815468 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:37.815474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:37.816135 (+ 661us) server_negotiation.cc:407] Connection header received
0811 01:50:37.816482 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:37.816488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:37.816592 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:37.816659 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:50:37.817506 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:37.818623 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:37.820992 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:37.821711 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:37.821809 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:37.822539 (+ 730us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:50:37.822678 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:50:37.822748 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:37.823025 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 01:50:37.823122 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 01:50:37.828833 16766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:37.814620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34476 (local address 127.21.214.125:38023)
0811 01:50:37.815013 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:37.815022 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:37.820779 (+ 5757us) server_negotiation.cc:407] Connection header received
0811 01:50:37.821173 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:37.821183 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:37.821301 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:37.821383 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:50:37.822243 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:37.823422 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:37.825760 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:37.826523 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:37.826666 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:37.827205 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:50:37.827362 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:50:37.827460 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:37.828320 (+ 860us) server_negotiation.cc:299] Negotiation successful
0811 01:50:37.828418 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246}
I20260811 01:50:37.828987 16780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 01:50:37.815224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60084 (local address 127.21.214.126:44133)
0811 01:50:37.817712 (+ 2488us) server_negotiation.cc:206] Beginning negotiation
0811 01:50:37.817720 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 01:50:37.821704 (+ 3984us) server_negotiation.cc:407] Connection header received
0811 01:50:37.822031 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 01:50:37.822039 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 01:50:37.822272 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 01:50:37.822455 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 01:50:37.823237 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:37.824320 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:37.826315 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 01:50:37.826893 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 01:50:37.827017 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 01:50:37.827615 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 01:50:37.827975 (+ 360us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 01:50:37.828110 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 01:50:37.828499 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 01:50:37.828602 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2317,"thread_start_us":2209,"threads_started":1}
I20260811 01:50:37.835646 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 16365
I20260811 01:50:37.836758 16437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:50:37.951825 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 16365
I20260811 01:50:37.981702 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 16451
I20260811 01:50:37.982968 16522 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:50:38.093287 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 16451
I20260811 01:50:38.119050 22361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw71eKA/build/asan/bin/kudu pid 16537
I20260811 01:50:38.120296 16605 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 01:50:38.232636 22361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw71eKA/build/asan/bin/kudu with pid 16537
2026-08-11T01:50:38Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5430 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10722 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (402014 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST